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March  30,  1999 


Mr.  Michael  F.  DiMario 
The  Public  Printer 
The  Government  Printing  Office 
North  Capitol  and  H  Sts.  NW 
Washington,  D.C.  20401 

Dear  Mr.  DiMario, 

It  is  with  great  pleasure  that  I  forward  herewith  a  copy  of  the  Final  Report  prepared  by  Westat,  Inc., 
the  contractor  selected  by  the  Government  to  undertake  Phase  II  of  the  three-part  study  called  "Assessment 
of  Electronic  Government  Information  Products."  As  you  requested,  the  U.S.  National  Commission  on 
Libraries  and  Information  Science  (NCLIS)  planned  and  implemented  this  research  survey,  pursuant  to  an 
interagency  agreement  between  NCLIS  and  the  Government  Printing  Office  (GPO),  approved  by  the  Joint 
Committee  on  Printing  (JCP). 

This  report  follows  on  the  process  begun  with  the  congressional  requirement,  contained  in  the  Senate 
Report  on  H.R.  1854,  the  FY  1996  Legislative  Branch  Appropriations  Act  (P.L.  104-53),  to  identify  the 
measures  necessary  for  a  successful  transition  to  a  more  electronic  Federal  Depository  Library  Program. 
That  requirement  resulted  in  a  study  published  by  the  Government  Printing  Office  in  June  1996.  There  was 
a  consensus,  however,  that  additional  work  was  required  (1)  to  identify  the  electronic  formats  and  mediums 
used  and/or  planned  by  Federal  publishing  entities,  and  (2)  to  determine  whether  public  or  private  sector 
standards  do,  or  could,  play  a  stronger  role  in  reducing  the  unnecessary  proliferation  of  these  formats  and 
mediums.  These  questions  precipitated  this  survey. 

I  am  extremely  pleased  to  note  that  the  survey  enjoyed  the  active  support  and  participation  of  all 
three  branches  of  Government.  Twenty-four  different  Federal  entities  participated,  including  the  Supreme 
Court,  several  committees  of  the  Congress,  one  regulatory  commission  and  19  Executive  Branch  agencies, 
including  most  of  the  Cabinet  Departments.  In  addition  to  this  broad  and  diverse  Federal  involvement  in 
the  survey,  an  impressive  74  percent  of  the  survey  forms  sent  to  the  agencies  were  returned  completed.  I 
believe  this  level  of  interest  and  support  is  highly  unusual,  and  could,  perhaps,  be  construed  as  a  reflection 
of  agency  desires  to  help  establish  a  systematic  baseline  for  measuring  and  monitoring  the  rapidly  changing 
and  evolving  kinds  and  mix  of  preferred  mediums,  formats,  and  standards. 

Our  representatives  and  your  staff  have  been  in  close,  harmonious  contact  from  the  earliest  stages  of 
planning  for  the  survey,  right  up  until  the  final  stages  of  review  of  the  final  report.  I  want  to  take  this 
opportunity  to  thank  especially  both  the  former  and  present  Superintendents  of  Documents,  as  well  as  the 
staffs  of  the  present  and  former  directors  of  the  Library  Programs  Service,  and  the  Office  of  Electronic 
Information  Dissemination  Service,  for  the  superb  support  NCLIS  and  the  contractor  received  throughout 
the  process. 


I  also  want  to  recognize  the  key  role  played  by  Forest  Woody  Horton,  Jr.  As  consultant  to  NCLIS, 
Woody's  broad  knowledge  of  how  Government  works  and  his  deep  understanding  of  Information 
Resources  Management  helped  to  move  the  study  along  most  effectively. 

Finally,  I  would  like  to  recognize  the  support  of  Vice-Chair  Martha  B.  Gould,  Commissioners  C.E. 
("Abe")  Abramson  who  chairs  the  NCLIS  Access  to  Government  Information  Committee,  Joan  R. 
Challinor,  and  Jose-Marie  Griffiths,  all  of  whom  have  been  staunch  advocates  throughout.  I  believe  you 
are  also  aware  of  the  strong  interest  and  support  NCLIS  Executive  Director  Robert  S.  Willard  personally 
accorded  this  study,  beginning  very  early  with  his  tenure  as  a  commissioner  and  extending  to  the  present 
day. 

The  long  review  and  analysis  process  of  the  contractor's  statistical  tabulations,  findings,  and 
observations  has  just  begun.  This  demanding  process  will  take  some  time,  in  part  because  the  number  of 
interested  communities  is  so  large,  and  in  part  because  the  subject  matter  is  so  technical,  involving  the  full 
range  of  information  handling  formats,  mediums,  and  standards,  and  quite  diverse  agency  plans  and 
practices.  Ultimately,  actions  needed  to  be  taken  will  most  likely  involve  new  or  strengthened  policies, 
rules,  and  regulations,  as  well  as  the  adoption  of  technical  standards,  some  of  which  could  have  legislative 
ramifications. 

It  is  now  the  Commission's  intention  to  begin  Phase  III.  We  will  take  the  now  completed  Phase  II 
Westat  report,  as  well  as  the  Phase  I  report  completed  in  1997  by  the  National  Academy  of  Sciences,  as 
points  of  departure.  They  will  be  reviewed  and  we  will  determine  if  additional  fact  gathering  is  required. 
We  can  then  move  forward  to  draw  conclusions  and  make  recommendations  to  the  Congress  and  the 
President  from  the  multitude  of  facts  and  expert  opinions  received  thus  far. 

NCLIS  will  continue  to  consult  with  GPO,  along  with  various  knowledgeable  individuals, 
interagency  and  special  advisory  groups,  all  of  whom  have  been  assisting  us  throughout  the  Phase  I  and  II 
efforts,  as  we  prepare  a  plan  for  the  Phase  III  initiative.  My  hope  is  that  we  will  keep  most  of  the  broader 
advisory  team  we  have  utilized  thus  far  in  place  until  we  have  completed  Phase  III. 

Finally,  I  want  to  thank  you  for  your  personal  leadership,  without  which  we  could  have  never  moved 
ahead  with  this  complex,  landmark  task. 


Sincerely  yours, 


Jeanne  Hurley  Simon 
Chairperson 
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Executive 
Summary 


The  Federal  Depository  Library  Program  (FDLP)  has  served 
and  continues  to  serve  the  American  public  by  ensuring 
localized  access  to  Federal  Government  information.  The 
mission  continues  to  be  as  important  today  to  the  fundamental 
success  of  our  democracy  as  it  was  when  the  FDLP  was  created. 
The  FDLP 's  original  mandate,  to  assist  Americans  regardless 
of  economic,  education,  or  geographic  considerations,  is  one 
that  must  not  be  lost  as  we  strategically  and  thoughtfully  use  the 
tools  of  the  electronic  age  to  enhance  that  mandate. 

Letter  to  Michael  F.  DiMario,  the  Public  Printer,  from  Senators 
John  Warner  and  Wendell  Ford  of  the  Senate  Committee  on 
Rules  and  Administration,  May  24,  1996. 


Background 


Congress  established  the  antecedents  to  the  Federal  Depository  Library 
Program  (FDLP)  in  the  Act  of  1 8 1 3  to  ensure  that  the  American  public  has 
access  to  its  Government's  information.  The  mission  of  the  FDLP,  part  of 
the  Superintendent  of  Documents  (SuDocs)  in  the  Government  Printing 
Office  (GPO),  is  to  assure  current  and  permanent  public  access  to  the 
universe  of  information  published  by  the  U.S.  Government.  Depository 
libraries  safeguard  the  public's  right  to  know  by  collecting,  organizing, 
maintaining,  preserving,  and  assisting  users  with  information  from  the 
Federal  Government.  GPO  provides  that  information  at  no  cost  to 
designated  depository  libraries  throughout  the  country.  These  depository 
libraries,  in  turn,  provide  local,  no-fee  access  to  Government  information  in 
all  formats  in  an  impartial  environment  with  professional  assistance.  Any 
member  of  the  public  can  visit  these  depository  libraries  and  use  the  Federal 
depository  collections. 

In  order  to  administer  the  FDLP,  as  required  by  the  enabling  legislation  for 
the  program,  44  U.S.C.  Chapter  19,  the  SuDocs  is  responsible  for  the 
acquisition,  classification,  format  conversion,  dissemination,  and 
bibliographic  control  of  tangible  and  electronic  Government  information 
products;  the  inspection  of  depository  libraries;  and  the  continuing 
education  and  training  initiatives  that  strengthen  the  ability  of  depository 
library  personnel  to  serve  the  public.  An  emerging  new  responsibility  is  to 
ensure  that  electronic  Government  information  products  disseminated 
through  the  FDLP,  or  incorporated  in  the  FDLP  Electronic  Collection, 
remain  permanently  accessible  to  the  public.  Under  44  U.S.C,  Sections 
1901-1903,  and  Office  of  Management  and  Budget  (OMB)  Circular  A- 130, 
Management  of  Federal  Information  Resources,  Federal  agencies  should 
make  all  their  publications  in  all  formats  available  to  SuDocs  for 
distribution  to  depository  libraries. 
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This  study  to  assess  electronic  medium  and  format  standards  for  the 
creation  and  dissemination  of  electronic  information  products  is  an  essential 
step  toward  ensuring  a  successful  and  cost-effective  transition  to  a  more 
electronic  FDLP.  The  three  goals  of  this  assessment  were  to: 

•  Identify  medium  and  format  standards  that  are  the  most  appropriate  for 
permanent  public  access; 

•  Assess  the  cost-effectiveness  and  usefulness  of  various  alternative 
medium  and  format  standards;  and 

•  Identify  public  and  private  medium  and  format  standards  that  are,  or 
could  be  used  for  products  throughout  their  entire  information  life 
cycle,  not  just  at  the  dissemination  or  permanent  public  access  stage. 

The  Superintendent  of  Documents  will  use  the  results  of  this  work  effort  to 
continue  to  plan  and  implement  the  transition  to  a  more  electronic  FDLP. 
The  five  major  specific  objectives  are: 

•  First,  with  respect  to  electronic  publishing  practices  and  plans  for 
Federal  agencies  (including  ways  in  which  the  FDLP  can  best 
accommodate  them),  the  objective  is  to  provide  an  analysis  of  current 
practices  as  well  as  future  plans  for  creating,  disseminating,  and 
providing  permanent  public  accessibility  to  electronic  information 
products,  and  to  identify  the  standards  for  software  and  electronic 
mediums  and  formats  that  are  used  throughout  the  product's 
information  life  cycle,  from  creation  to  archiving  but  especially  at  the 
stage  of  dissemination  for  permanent  public  access. 

•  Second,  with  respect  to  cost-effectiveness  of  various  dissemination 
mediums  and  formats  that  are,  or  could  be  utilized,  the  objective  is  to 
gather  information  on  standards  (whether  mandated  or  consensual)  that 
will  assist  the  FDLP  in  making  near-term  decisions  regarding  the  cost- 
effectiveness  of  alternative  mediums  and  formats  for  all  FDLP 
participants.  This  information  should  also  assist  participants  in  long- 
term  planning  for  permanent  public  accessibility,  and  the  collection  and 
analysis  of  overall  information  life  cycle  costs. 

•  Third,  with  respect  to  the  practical  utility  of  various  electronic 
mediums  and  formats  to  depository  libraries  and  the  public,  the 
objective  is  to  identify  preferred  standards  used  in  various  mediums 
and  formats  that  depository  libraries  will  need  to  support. 

•  Fourth,  with  respect  to  utilizing  standards  employed  in  mediums  and 
formats  that  can  be  used  throughout  all  stages  of  the  information  life 
cycle  (including  creation,  composition,  computer  terminal  display, 
encryption,  secure  digital  signature  with  non-repudiation,  and  secure 
transmission  capabilities),  for  electronic  dissemination,  but  especially 
permanent  public  accessibility,  the  objective  is  to  assess  standards  for 
basic  security  services  in  order  to  provide  for  secure  and  reliable 
transmission  and  document  interchange. 
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•  Fifth,  with  respect  to  standards  that  are  being  developed  and  used  in  the 
private  sector,  the  objective  is  to  identify  existing  and  planned 
standards  for  the  purpose  of  determining  what  the  FDLP  must  do  to 
accommodate  their  adoption  in  terms  of  hardware/software 
requirements,  staff  and  user  education  and  training,  and  budgetary 
impacts. 


The  study  utilized  both  quantitative  and  qualitative  data  collection 
activities:  a  survey  of  a  cross-section  of  314  Government  information 
products  from  24  agencies  and  interviews  with  experts.  The  response  rate 
for  the  survey  was  74  percent.  This  cross-section  of  products  was  not  a 
randomly  selected  sample  due  to  cost  and  time  constraints.  Instead. 
NCLIS  and  GPO — assisted  by  various  groups,  including  the  library 
associations  represented  by  the  Inter-Association  Working  Group  on 
Government  Information  Policy  (IAWG),  the  Federal  Library  and 
Information  Center  Committee  (FLICC),  the  Depository  Library  Council 
(DLC),  and  the  Interagency  Council  on  Printing  and  Publication  Services 
(ICPPS) — developed  and  refined  the  criteria  for  product  selection.  NCLIS, 
GPO,  and  the  other  organizations  asked  knowledgeable  members  of  these 
groups  to  identify  products  that  met  one  or  more  of  six  criteria. 

NCLIS  distributed  the  list  of  preliminary  products  to  agency  Chief 
Information  Officers  (CIOs)  who  were  asked  to  validate  and  coordinate  the 
final  selections  with  their  appropriate  agency  personnel.  In  addition. 
NCLIS  asked  CIOs  to  select  an  agency  coordinator.  The  coordinator's  role 
was  to  oversee  the  distribution  of  product  questionnaires  to  the  appropriate 
respondents  and  to  encourage  respondents  to  complete  the  questionnaire  and 
return  it  to  Westat. 

Product  selection  was  based  on  six  criteria: 

•  Increased  emphasis  on  electronic  dissemination,  rather  than 
continuation  of  paper  and  microform  dissemination; 

•  Replacement  of  older  electronic  mediums  and  formats  with  state-of-the- 
art  technologies; 

•  Adoption  of  mandated  (Government  or  private  sector)  and  consensual 
(common  agency  practice)  medium  and  format  standards; 

•  Adoption  and  use  of  preferred  mediums  or  formats  that  have 
widespread  support  from  agency,  depository  library,  and  user 
communities; 

•  Exemplified  cost-effective  mediums  and  standards,  especially  those  that 
can  be  used  throughout  the  entire  information  life  cycle,  rather  than  the 
use  of  expensive  customized  or  shelf  packages;  and 
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•  Exemplified  awareness  of  the  important  impact  of  medium  and  format 
decisions  on  permanent  accessibility,  authentication,  and/or  security 
encryption  protection. 

The  survey  requested  information  on  four  main  topics: 

•  General  information  about  the  product  and  agency  that  produced  it. 

•  The  product's  current  profile  including  the  kinds  of  data  the  product 
contains,  mediums  in  which  it  is  produced,  formats  and  online 
approaches  used  (if  applicable);  and  searchability  and  retrievability  of 
the  product. 

•  Future  plans  for  the  product  including  changes  in  its  data,  mediums, 
and  formats. 

•  Other  issues  including  metadata,  permanent  public  access,  permanent 
retention,  authenticity,  updating/upgrading  plans,  user  fees,  licensing, 
and  public  domain. 

The  qualitative  data  collection  included  site  visits  to  three  depository 
libraries,  meetings  with  representatives  of  five  Government  agencies,  and 
telephone  interviews  with  six  experts.  The  qualitative  data  collection 
included  site  visits,  agency  meetings,  and  expert  interviews.  Westat 
conducted  site  visits  to  three  Federal  depository  libraries: 

•  McKeldin  Library,  University  of  Maryland  College  Park,  College  Park, 
Maryland 

•  Washington  College  of  Law  Library,  American  University, 
Washington,  D.C. 

•  Montgomery  County  Rockville  Regional  Public  Library,  Rockville, 
Maryland 

The  purpose  of  the  visits  was  to  discuss  the  effects  of  the  transition  to  a 
more  electronic  Federal  Depository  Library  Program  on  the  end  user  and  on 
the  services  and  resources  of  each  library. 

Meetings  with  agency  representatives  had  a  twofold  purpose: 

•  To  collect  qualitative  data  about  electronic  Government  information 
products,  such  as  cost-effectiveness  of  standards,  use  of  locator  tools, 
results  of  user  surveys,  etc.,  that  were  not  covered  in  the  survey;  and 

•  To  discuss  the  procedures  for  distribution  of  the  questionnaire. 

In  addition  to  inviting  agency  coordinators  and  respondents,  the  statement 
of  work  specified  that  Westat  invite  representatives  of  the  following  offices 
to  attend  the  meetings: 
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•  Public  affairs  or  communications  offices, 

•  Agency  printing  and  publishing  units, 

•  Information  technology  or  electronic  information  systems  offices, 

•  Agency  libraries,  and 

•  Relevant  program  offices. 

The  following  six  agencies  agreed  to  schedule  a  meeting:  Department  of 
Health  and  Human  Services,  Department  of  Education,  U.S.  Supreme 
Court,  Department  of  Commerce,  Environmental  Protection  Agency,  and 
the  National  Archives  and  Records  Administration.  Only  four  of  the  six 
agencies  chose  to  discuss  the  qualitative  questions  at  the  meeting.  The  other 
two  agencies  discussed  the  questionnaire  only  and  agreed  to  respond  to  the 
discussion  questions  in  writing,  although  only  one  actually  submitted  their 
written  questions. 

Finally,  Westat  held  four  telephone  interviews  with  six  content  experts. 
The  experts  included  two  webmasters  (Linda  Wallace  from  the  Internal 
Revenue  Service,  and  Jerry  Malitz  from  the  National  Center  for  Education 
Statistics);  two  preservation  specialists  (Evelyn  Frangakis  from  the 
National  Agricultural  Library,  and  Abby  Smith  from  the  Council  on 
Library  and  Information  Resources);  and  two  professors  in  information 
resources  management  (John  Bertot  and  Charles  McClure).  The  purpose  of 
expert  interviews  was  to: 

•  Solicit  opinions  of  experts  on  topics  not  adequately  covered  on  the 
survey  or  in  the  agency  meetings, 

•  Ask  questions  to  provide  a  broader  context  in  which  to  view  the  issues, 
and 

•  Explore  current  initiatives  and  future  directions. 


Key  Findings  These  findings  reflect  the  major  results  of  the  survey  and  qualitative  data 

collection: 

Policy  and  Planning  Issues 

1.  There  is  an  overall  lack  of  Government  information  policy  guiding 
electronic  publishing,  dissemination,  permanent  public  access,  or 
information  life  cycle  management,  especially  as  information  policy 
relates  to  agency  missions.  Also,  there  is  a  lack  of  overall  coordination 
of  these  initiatives  at  the  Governmental,  branch,  or  even  agency  level 
(pp.  68-69). 

2.  Responsibility  for  electronic  publishing  within  agencies  is  decentralized, 
diffuse,     and    unclear.        Some    agencies    either    could  not 
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identify  or  had  difficulty  identifying  the  proper  respondent  within  their 
own  agency,  or  even  the  person  who  was  responsible  for  the  product 
(pp.  11  and  14). 

3.  Some  Government  agencies  are  monitoring  the  information  needs  of 
their  users  to  enhance  current  access  to  electronic  Government 
information  products  (p.  65). 

4.  There  is  a  lack  of  specific  planning  for  product  development  and 
technological  migration  (pp.  34-36;  table  23  on  p.  42). 

5.  There  is  a  lack  of  planning  for  or  consideration  of  web  design 
approaches  that  comply  with  the  Americans  with  Disabilities  Act 
(ADA)  (table  6a,  p.  29). 

Permanent  Public  Access 

6.  The  concept  of  permanent  public  access  (PPA)  is  not  well  understood. 
Respondents  also  had  difficulty  distinguishing  between  PPA  for 
electronic  products  and  archiving  electronic  Federal  records  with  the 
National  Archives  and  Records  Administration  (tables  18-20,  pp.  39- 
40). 

7.  Metadata  and  their  importance  to  public  access  are  not  well  understood, 
particularly  as  they  may  affect  PPA.  Only  27  percent  of  respondents 
reported  having  a  metadata  record  for  the  products  surveyed  (table  19, 
p.  39). 

8.  For  some  products,  PPA  results  from  the  agencies'  use  of  a  host 
disseminator,  such  as  GPO  Access  (p.  1 1). 

Authenticity 

9.  There  is  a  lack  of  understanding  of  what  ensuring  authenticity  entails, 
and  a  lack  of  planning  for  or  consideration  of  ensuring  authenticity  of 
electronic  Government  information  products  (table  21,  p.  41). 

Product  Characteristics 

10.  Fifteen  percent  of  the  products  surveyed  are  not  in  the  public  domain, 
for  all  or  part  of  the  product  (table  27,  p.  45).  In  addition,  user  fees  are 
charged  for  30  percent  of  the  products  (table  24,  p.  43). 

1 1 .  The  most  prevalent  types  of  mediums  are  the  web,  paper,  CD-ROM, 
and  bulletin  board  systems  (table  3a,  p.  22);  the  most  prevalent  formats 
are  HTML,  PDF,  GIF,  JPEG,  TIFF,  and  ASCII  (table  4a,  p.  25). 

12.  The  most  prevalent  types  of  data  contained  in  the  products  surveyed  are 
textual,  numerical,  bibliographic,  and  graphical  (tables  2a  and  2b,  p. 
20). 


xvi 


Assessment  of  Electronic  Government  Information  Products 


Standards 

13.  There  is  a  lack  of  standardization  for  producing  Government 
information  products  on  CD-ROM  (e.g.,  installation  instructions,  user 
documentation)  (p.  55). 

14.  The  most  prevalent  medium  and  format  standards  identified  in  the 
survey  are  common  agency  practice  rather  than  agency-mandated 
(tables  3b,  4b,  6b,  pp.  23,  26,  and  30). 

15.  Some  Government  agencies  have  established  guidelines  or  best 
practices  for  presenting  and  organizing  Government  information 
products  on  the  web,  although  full  compliance  with  the  guidelines  is  a 
goal  that  has  not  yet  been  achieved  (p.  64). 

16.  Some  Government  agencies  are  exploring  a  range  of  innovative  formats 
and  web  design  approaches  for  electronic  Government  information 
products  (p.  57). 


As  a  followup  effort,  NCLIS  indicated  that  they  will  use  these  findings  as  a 
point  of  departure  and  analyze  them  in  greater  depth.  It  is  expected  that  this 
followup  effort  will  result  in  broad  conclusions  and  recommendations  to  the 
President  and  Congress  about  how  the  problems  and  challenges  revealed  in 
this  study  can  be  constructively  addressed  to  improve  current  and  future 
public  access  to  electronic  Government  information. 
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Since  1813,  the  American  public  has  benefited  from  the  ability  to  gain  free 
access  to  Federal  Government  information.  This  unique  American  right  to 
no-fee  access  to  Government  information  is  made  possible  through  the 
Federal  Depository  Library  Program  (FDLP)  of  the  Superintendent  of 
Documents  (SuDocs)  in  the  Government  Printing  Office  (GPO).  The 
FDLP  has  significantly  contributed  to  creating  an  informed,  educated,  and 
culturally  enriched  U.S.  citizenry. 

This  introduction  provides  a  brief  overview  of  the  FDLP  and  background 
information  on  the  purpose  and  objectives  of  this  study  to  assess  electronic 
Government  information  products. 

The  mission  of  the  Federal  Depository  Library  Program  is  to  assure  current 
and  permanent  public  access  to  the  universe  of  information  published  by  the 
U.S.  Government.  The  FDLP  was  established  by  Congress  to  ensure  that 
the  American  public  has  access  to  its  Government's  information. 
Depository  libraries  safeguard  the  public's  right  to  know  by  collecting, 
organizing,  maintaining,  preserving,  and  assisting  users  with  information 
from  the  Federal  Government.  The  Government  Printing  Office  provides 
Government  information  at  no  cost  to  designated  depository  libraries 
throughout  the  country.  These  depository  libraries,  at  their  own  expense, 
provide  local,  no-fee  access  to  Government  information  in  all  formats  in  an 
impartial  environment  with  professional  assistance.  Any  member  of  the 
public  can  visit  these  depository  libraries  and  use  the  Federal  depository 
collections. 

Products  distributed  by  GPO  for  depository  library  collections  include  all 
electronic  Government  information  products  that  are  of  public  interest  or 
educational  value.  By  law,  the  FDLP  excludes  those  products  that  are 
solely  for  administrative  or  operational  purposes,  classified  for  reasons  of 
national  security,  or  the  use  of  which  is  constrained  by  privacy 
considerations  (GPO,  1998,  p.  4). 

In  order  to  administer  the  FDLP,  as  required  by  the  enabling  legislation  for 
the  program,  44  U.S.C.  Chapters  17,  19,  and  41,  the  SuDocs  is  responsible 
for  the  acquisition,  classification,  format  conversion,  dissemination,  and 
bibliographic  control  of  tangible  and  electronic  Government  information 
products;  the  inspection  of  depository  libraries,  and  the  continuing 
education  and  training  initiatives  that  strengthen  the  ability  of  depository 
library  personnel  to  serve  the  public.  An  emerging  new  responsibility  is  to 
ensure  that  electronic  Government  information  products  disseminated 
through  the  FDLP,  or  incorporated  in  the  FDLP  Electronic  Collection, 
remain  permanently  accessible  to  the  public.  Under  44  U.S.C,  Sections 
1901-1903,  and  Office  of  Management  and  Budget  (OMB)  Circular  A- 130, 
Management  of  Federal  Information  Resources,  Federal  agencies  should 
make  all  their  publications  in  all  produced  formats  available  to  SuDocs  for 
distribution  to  depository  libraries. 


1 Introduction 
and 
Background 


The  Federal 
Depository 
Library  Program 


1 


Assessment  of  Electronic  Government  Information  Products 


How  the  Federal 
Depository  Library 
Program  Works 


GPO  provides  Government  information  at  no  cost  to  designated  depository 
libraries  throughout  the  country.  These  depository  libraries,  at  their  own 
expense,  provide  local,  no-fee  access  with  professional  assistance  to  this 
information  in  all  formats.  Access  to  Federal  Government  information  is 
available  through  more  than  1,350  depository  libraries  located  throughout 
United  States  and  its  territories.  Fifty-three  of  the  depositories  are  regionals, 
and  the  remaining  are  selective  depositories.  The  regional  libraries  receive 
and  maintain  everything  that  is  distributed  through  the  program,  unless  they 
are  superseded.  The  selective  libraries  pre-select  the  types  of  publications 
they  wish  to  receive  based  on  the  specific  needs  and  interests  of  the 
communities  they  serve.  Of  the  libraries  in  the  FDLP,  approximately  50 
percent  are  academic,  20  percent  are  public,  1 1  percent  are  law,  5  percent 
are  community  college,  4  percent  are  Federal  agency,  and  10  percent  are 
special,  state,  court,  and  Federal  court  libraries. 


Before  the  evolution  of  electronic  publishing  media,  especially  the  Internet, 
Federal  Government  agencies  published  information  almost  exclusively  in  a 
centralized  print  environment  that  facilitated  easy  distribution  to  the  Federal 
depository  libraries.  Now,  Federal  Government  agencies  are  doing  their 
own  electronic  publishing  and  creating  and  managing  their  own  websites  to 
disseminate  a  variety  of  Government  information  products.  This  study 
resulted  from  Congress's  concerns  about  the  short-  and  long-term  effects  of 
electronic  publishing  on  the  ability  of  all  U.S.  citizens  to  continue  to  gain 
affordable  and  easy  access  to  Government  information. 


Background 
of  the  Study 


This  study  to  assess  electronic  Government  information  products  was 
authorized  by  the  Joint  Committee  on  Printing  and  was  sponsored  by  the 
Superintendent  of  Documents,  U.S.  Government  Printing  Office.  The 
initial  need  for  this  project  was  identified  in  GPO's  cooperative  1996  Study 
to  Identify  Measures  Necessary  for  a  Successful  Transition  to  a  More 
Electronic  Federal  Depository  Library  Program.  This  study  (see 
www. access. gpo.gov/su_docs/dpos/fdlppubs. html#4)  was  conducted  at  the 
direction  of  Congress.  In  order  to  conduct  the  study,  the  Public  Printer 
established  a  working  group  consisting  of  representatives  from  the 
following  program  stakeholders  and  constituents: 


GPO, 

Appropriate  congressional  committees, 

Congressional  Research  Service  at  the  Library  of  Congress, 

Office  of  Management  and  Budget, 

National  Archives  and  Records  Administration, 

Federal  Publishers  Committee, 
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•  Interagency  Council  on  Printing  and  Publication  Services  (ICPPS), 

•  Administrative  Office  of  the  U.S.  Courts,  and 

•  Depository  library  community. 

One  of  the  committee's  major  recommendations  was  to  assess  electronic 
medium1  and  format  standards  for  the  creation  and  dissemination  of 
electronic  information  products.  The  committee  considered  this  assessment 
an  essential  step  toward  ensuring  a  successful  and  cost-effective  transition 
to  a  more  electronic  FDLP. 


Project  Phases  This  project  is  being  undertaken  in  three  phases.   The  first  phase  of  the 

project  consisted  of  a  review  by  the  National  Academy  of  Science's 
Computer  Science  and  Telecommunications  Board  (CSTB)  in  which  CSTB 
developed  a  detailed  statement  of  work  that  defined  the  data  collection 
process  required  to  conduct  the  assessment  (see 
http://www.nclis.gov/info/gpol.html). 

This  report  is  a  product  of  Phase  II  of  the  project.  GPO  commissioned  the 
National  Commission  on  Libraries  and  Information  Science  (NCLIS)  to 
undertake  a  survey  and  assessment  of  electronic  Government  information 
products.  NCLIS  awarded  the  contract  to  Westat,  a  survey  research 
company,  to  undertake  research  and  data  collection  from  Federal  agencies 
in  all  three  Branches,  as  well  as  solicit  the  opinions  of  selected 
knowledgeable  experts.  The  contract  further  called  for  Westat  to  complete 
an  analysis  of  the  data  and  expert  opinions  for  the  purpose  of  interpreting 
their  general  meaning  and  significance,  including  identifying  broad 
emerging  trends  and  patterns,  and  documenting  findings. 

In  Phase  III,  NCLIS  will  identify  an  appropriate  organization  to  review 
Phase  I  and  Phase  II  findings,  as  well  as  to  review  the  data  and  develop 
conclusions  and  recommendations  for  GPO,  the  Congress,  and  the 
President. 


1  The  word  medium  is  used  throughout  this  report.  It  refers  to  the  physical,  chemical,  or  biological 
substrate  used  to  create,  organize,  store,  search  for,  retrieve,  disseminate,  or  permanently  archive  data, 
documents  or  literature,  including  paper,  microforms,  fiber  optic  cables,  photographic  film,  CD-ROMs, 
floppy  diskettes,  magnetic  storage  devices,  sound  recordings,  and  videotape.  The  term  media  is  not  used 
in  this  report  as  the  plural  of  "medium."  Media  is  the  means  used  to  publish,  communicate,  disseminate, 
and  distribute  information,  regardless  of  format,  such  as  radio,  television,  magazines,  office  or  home 
PCs,  scholarly  journals,  and  videotape. 
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Study  Goals  and  Information   gathered   from   this   assessment   will   be   used   by  the 

Objectives  (Phase  II)  Superintendent  of  Documents  to  facilitate  improved  public  access  to 

Federal  Government  information  made  available  to  Federal  depository 
libraries  and  the  general  public  through  the  FDLP.  More  specifically,  for 
this  cross-section  of  Government  information  products,  the  Phase  II  goals 
were  to: 

•  Identify  medium  (see  glossary  in  Appendix  E  for  the  difference  between 
the  medium  and  media)  and  format  standards  that  are  the  most 
appropriate  for  permanent  public  access, 

•  Assess  the  cost-effectiveness  and  usefulness  of  various  alternative 
medium  and  format  standards,  and 

•  Identify  public  and  private  medium  and  format  standards  that  are,  or 
could  be  used  for  products  throughout  their  entire  information  life 
cycle,  not  just  at  the  dissemination  or  permanent  public  access  stage. 

The  Superintendent  of  Documents  will  use  the  results  of  this  work  effort  to 
continue  to  plan  and  implement  the  transition  to  a  more  electronic  FDLP. 
The  five  major  specific  objectives  are: 

•  First,  with  respect  to  electronic  publishing  practices  and  plans  of 
Federal  agencies  (including  ways  in  which  the  FDLP  can  best 
accommodate  them),  the  objective  is  to  provide  an  analysis  of  current 
practices  as  well  as  future  plans  for  creating,  disseminating,  and 
providing  permanent  public  accessibility  to  electronic  information 
products,  and  to  identify  the  standards  for  software  and  electronic 
mediums  and  formats  that  are  used  throughout  the  product's 
information  life  cycle,  from  creation  to  archiving,  but  especially  at  the 
stage  of  dissemination  for  permanent  public  access. 

•  Second,  with  respect  to  cost-effectiveness  of  various  dissemination 
mediums  and  formats  that  are,  or  could  be  utilized,  the  objective  is  to 
gather  information  on  standards  (whether  mandated  or  consensual)  that 
will  assist  the  FDLP  in  making  near-term  decisions  regarding  the 
cost-effectiveness  of  alternative  mediums  and  formats  for  all  FDLP 
participants.  This  information  should  also  assist  participants  in 
long-term  planning  for  permanent  public  accessibility,  and  the 
collection  and  analysis  of  overall  information  life  cycle  costs. 

•  Third,  with  respect  to  the  practical  utility  of  various  electronic 
mediums  and  formats  to  depository  libraries  and  the  public,  the 
objective  is  to  identify  preferred  standards  used  in  various  mediums 
and  formats  that  depository  libraries  will  need  to  support. 

•  Fourth,  with  respect  to  utilizing  standards  employed  in  mediums  and 
formats  that  can  be  used  throughout  all  stages  of  the  information  life 
cycle  (including  creation,  composition,  computer  terminal  display, 
encryption,  secure  digital  signature  with  non-repudiation  and  secure 
transmission   capabilities),    but   especially   for   permanent  public 
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accessibility,  the  objective  is  to  assess  standards  for  basic  security 
services  in  order  to  provide  for  secure  and  reliable  transmission  and 
document  interchange. 

•  Fifth,  with  respect  to  standards  that  are  being  developed  and  used  in  the 
private  sector,  the  objective  is  to  identify  existing  and  planned 
standards  for  the  purpose  of  determining  what  the  FDLP  must  do  to 
accommodate  their  adoption  in  terms  of  hardware/software 
requirements,  staff  and  user  education  and  training,  and  budgetary 
impacts. 


Scope  and 
Organization 
of  the  Report 


The  primary  data  collection  activities  included  a  survey  and  interviews. 
Westat,  per  the  requirements  established  by  NCLIS  in  consultation  with 
GPO,  surveyed  a  cross-section  of  electronic  information  products  from 
Federal  agencies  in  all  three  branches  of  Government  and  solicited  the 
opinions  of  selected  knowledgeable  experts.  This  cross-section  of  products 
was  not  a  randomly  selected  sample  due  to  cost  and  time  constraints. 
Therefore,  readers  are  cautioned  about  generalizing  the  findings  to  all 
electronic  Government  information  products. 


Westat  surveyed  electronic  Government  information  products  to  determine 
the  mediums  and  formats  in  which  products  are  currently  produced  and  the 
standards,  if  any,  that  are  being  used.  The  survey  also  asked  respondents 
questions  about  the  agency's  future  plans  for  adding  or  changing  products, 
including  the  mediums  and  formats  in  which  they  will  be  disseminated  for 
permanent  public  access. 

This  report  is  limited  to  presenting  and  discussing  the  survey  findings  and 
findings  from  qualitative  site  visits,  agency  meetings,  and  expert  interviews. 
Phase  III  of  the  project  will  focus  on  drawing  conclusions  and 
recommendations  based  on  work  conducted  during  Phases  I  and  II. 


The  report  is  organized  in  five  parts:  introduction  and  background, 
methodology,  survey  analysis  and  findings,  qualitative  findings,  and 
discussion  of  quantitative  and  qualitative  findings.  Please  note  that 
Appendix  E  contains  a  glossary  of  terms  and  acronyms  used  on  the 
questionnaire  and  throughout  this  report. 
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Methodology  secon<^  Part  °f  tne  report  discusses  the  following  topics: 


•  The  process  of  selecting  a  cross-section  of  electronic  Government 
information  products, 

•  Agency  coordinator  briefings, 

•  Questionnaire  design  and  development, 

•  Nonresponse  and  data  retrieval  followup,  and 

•  The  methodology  for  the  qualitative  data  collection  activities,  i.e.,  site 
visits,  agency  meetings,  and  expert  interviews. 


Product  Selection 


NCLIS  and  GPO — assisted  by  various  groups,  including  the  library 
associations  represented  by  the  Inter-Association  Working  Group  on 
Government  Information  Policy  (IAWG),  the  Federal  Library  and 
Information  Center  Committee  (FLICC),  the  Depository  Library  Council 
(DLC),  and  the  Interagency  Council  on  Printing  and  Publication  Services 
(ICPPS) — developed  and  refined  a  set  of  criteria  for  product  selection. 
NCLIS,  GPO,  and  the  other  representatives  asked  knowledgeable  members 
of  these  groups  to  identify  products  that  met  one  or  more  of  the  following 
six  guidelines: 


Increased  emphasis  on  electronic  dissemination  rather  than  continuation 
of  paper  and  microform  dissemination; 

Replacement  of  older  electronic  mediums  and  formats  with  state-of-the- 
art  technologies; 

Adoption  of  mandated  (Government  or  private  sector)  and  consensual 
(common  agency  practice)  medium  and  format  standards; 

Adoption  and  use  of  preferred  mediums  or  formats  that  have 
widespread  support  from  agency,  depository  library,  and  user 
communities; 

Exemplified  cost-effective  mediums  and  standards,  especially  those  that 
can  be  used  throughout  the  entire  information  life  cycle,  rather  than  the 
use  of  expensive  customized  or  shelf  packages;  and 

Exemplified  awareness  of  the  important  impact  of  medium  and  format 
decisions  on  permanent  accessibility,  authentication,  and/or  security 
encryption  protection. 


The  products  were  not  randomly  selected;  therefore,  readers  are  cautioned 
about  generalizing  the  findings  to  all  electronic  Government  information 
products. 

In  April  1998,  NCLIS  distributed  the  preliminary  list  of  products  to  agency 
Chief  Information  Officers  (CIOs),  who  were  asked  to  validate  and 
coordinate  the  final  selections  with  appropriate  agency  personnel.  In 
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addition,  NCLIS  asked  CIOs  to  select  an  agency  coordinator.  The 
coordinator's  role  was  to  oversee  the  distribution  of  product  questionnaires 
to  the  appropriate  respondents  and  to  encourage  respondents  to  complete 
the  questionnaire  and  return  it  to  Westat.  (See  Appendix  B  for  a  list  of 
coordinators  who  participated  in  this  study.) 

The  final  product  list  included  328  products  from  24  agencies  (Appendix 
C).  Over  the  course  of  the  data  collection,  the  number  of  products 
decreased  from  328vto  3 14  for  the  following  reasons: 

•  Several  products  were  discontinued  and  no  longer  exist. 

•  Several  products  were  in  paper  only  and  agencies  had  no  plans  to 
migrate  them  to  an  electronic  medium;  therefore,  they  fell  outside  the 
scope  of  this  study. 

•  Agency  coordinators  could  not  identify  respondents  for  some  products, 
so  there  was  no  one  to  complete  the  questionnaire. 

•  Several  questionnaires  were  undeliverable  due  to  unknown  or  incorrect 
respondent  addresses;  no  alternate  respondent  could  be  located  in  a  few 
cases. 


Coordinator 
Briefings 


NCLIS  and  GPO  planned  and  conducted  two  coordinator  briefings  in  June 
and  July  1998,  and  asked  Westat  to  attend  them  (see  Appendix  A  for 
agenda).  The  purpose  of  these  briefings  was  to: 


Provide  an  overview  of  the  study  including  background,  purpose,  goals, 
and  schedule, 

Discuss  their  specific  tasks, 

Review  the  draft  questionnaire  with  coordinators  and  solicit  their  input 
on  changes, 

Collect  their  final  list  of  products,  and 

Thank  them  for  their  participation  and  cooperation. 


Coordinators  were  asked  to: 


Assist  Westat  in  pretesting  the  survey  instrument, 

Identify  and  brief  appropriate  internal  participating  offices, 

Identify  product  respondents  for  survey  followup, 

Schedule  and  participate  in  voluntary  agency  meetings  with  Westat, 

Distribute  questionnaires  to  agency  respondents, 

Ensure  timely  completion  and  submission  of  survey  instruments,  and 

Cooperate  with  Westat  on  followup. 
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Only  a  few  coordinators  brought  their  final  selections  to  the  agency 
meetings;  most  agencies  needed  much  more  time  to  review  and  finalize  their 
product  selections.  The  questionnaire  review  also  served  as  an  informal 
pretest  of  the  questionnaire. 


Questionnaire  NCLIS,  with  consultation  from  GPO,  developed  the  initial  five-page  list  of 

questions.    This  list  of  questions  was  included  as  an  appendix  to  the 

Design  statement  of  work.    Westat  worked  with  GPO  and  NCLIS  from  June 

through  July  to  expand  and  refine  the  list  of  questions  to  a  13 -page 
instrument  with  appropriate  instructions,  examples,  skip  patterns,  open- 
ended  questions,  please-specify  questions,  etc.  Westat  pretested  the 
questionnaire  informally  at  the  two  coordinator  briefings.  The  coordinators 
helped  Westat  to  clarify  some  questions,  expand  the  format  choices,  and 
add  a  few  more  questions. 

Westat  conducted  a  more  formal  pretest  with  personnel  from  six 
Government  agencies.  These  pretests  led  to  the  following  substantive 
changes  in  the  questionnaire: 

•  Clarification  of  instructions  and  wording  of  several  questions, 

•  Addition  of  more  format  options, 

•  Addition  of  the  definition  of  "product"  at  the  beginning  of  the 
questionnaire,  and 

•  Clarification  of  definitions  included  in  the  glossary. 

Westat,  with  final  approval  by  NCLIS  and  GPO,  finalized  the  questionnaire 
by  mid- August  1998.  (See  Appendix  D  for  the  cover  letters  and  Appendix 
E  for  the  final  questionnaire.) 


Distribution  of  the 
Questionnaires 


During  the  last  week  of  September  and  the  first  week  of  October,  Westat 
distributed  the  questionnaires  to  23  agencies  through  the  agency 
coordinators.  On  October  9,  1998,  NCLIS  requested  that  Westat  add  more 
products  to  the  survey  by  including  a  24th  agency,  the  Securities  and 
Exchange  Commission  (SEC). 


Westat  created  a  database  of  products  and  corresponding  coordinators  or 
respondents  and  their  addresses,  phone  and  fax  numbers,  and  e-mail 
addresses,  and  prepared  and  mailed  packets  to  the  agency  coordinators.  The 
agency  coordinators  were  responsible  for  ensuring  that  each  packet  was 
sent  to  the  appropriate  product  respondent  in  a  timely  fashion.  These 
packets  included  the  following  materials  for  each  product  that  was  to  be 
surveyed: 


Cover  letter  to  coordinator, 
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•  Cover  letter  to  respondent, 

•  Questionnaire, 

•  Glossary  of  terms  used  in  the  questionnaire,  and 

•  Postage-paid  return  envelope. 

A  few  agency  coordinators  requested  that  Westat  send  questionnaires 
directly  to  their  product  respondents  and  a  copy  of  the  respondents'  packets 
to  the  coordinators  themselves.  Westat  sent  questionnaire  materials  directly 
to  the  respondents  at  the  Department  of  Commerce,  the  Department  of  the 
Interior,  the  Executive  Office  of  the  President,  and  the  U.S.  Congress. 
These  respondent  packets  included  the  following  materials: 

•  Cover  letter  to  respondent, 

•  Questionnaire  for  each  product  he/she  was  assigned  to  survey, 

•  Glossary  of  terms  used  in  the  questionnaire,  and 

•  Postage-paid  return  envelope. 

In  addition,  the  Department  of  Health  and  Human  Services  asked  Westat  to 
send  an  e-mail  message  to  the  individual  product  respondents  notifying  them 
that  they  could  download  the  final  version  of  the  questionnaire  and  cover 
letters  from  the  PDF  file  located  on  the  NCLIS  website  at 
http://www.nclis.gov/news/nclisqux.pdf  in  order  to  complete  the 
questionnaire. 


Followup  for 
Non  response,  Data 
Retrieval,  and 
Inconsistency 


Westat  made  the  first  calls  for  nonresponse  to  agency  coordinators.  These 
calls  began  in  early  November  and  continued  through  mid-December.  In 
addition,  NCLIS  sent  periodic  coordinator  bulletins  to  keep  coordinators 
updated  on  the  progress  of  the  study  and  to  encourage  respondents — 
through  the  coordinators — to  complete  questionnaires  and  return  them  to 
Westat. 


Westat  began  a  second  round  of  nonresponse  followup  calls  to  respondents 
from  mid-December  through  the  end  of  January  1999.  From  mid- 
November  through  the  first  week  in  January  1999,  Westat  made  calls 
directly  to  respondents  for  data  retrieval  (i.e.,  missing  data)  and 
inconsistencies  (i.e.,  a  respondent  checked  "yes"  to  one  question,  but  the 
next  question  was  answered  in  a  way  that  suggested  a  "no"  answer  to  the 
first  question). 

Approximately  40  percent  of  the  questionnaires  required  some  type  of  data 
retrieval  followup  for  one  or  more  questions.  Some  questions,  such  as  16, 
18-19,     and     21a,     concerning     metadata,     permanent  retention, 
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authenticity,  and  the  product's  supporting  technology,  presented  particular 
problems.  Westat  added  a  "don't  know"  category  to  these  questions  as  a 
result  of  the  nonresponse  data  retrieval.  In  addition,  most  respondents 
skipped  questions  13d,  14d,  and  15d  about  long-term  plans  for  changing  the 
product.  Data  retrieval  phone  calls  and  discussions  with  agency 
coordinators  suggest  respondents  skipped  these  questions  because  agencies 
had  not  yet  developed  long-term  plans. 

The  calls  to  respondents  for  data  retrieval  and  data  inconsistency  revealed 
the  following  reasons  for  nonresponse: 

•  Did  not  know  the  answer. 

•  Could  not  identify  anyone  who  knew  the  answer. 

•  Did  not  understand  the  question  or  the  concept;  using  glossary  did  not 
help. 

•  Did  not  have  time  to  research  the  answer;  had  other  work  priorities. 

In  a  few  instances,  it  was  clear  that  the  agency  was  not  in  a  good  position  to 
respond  to  the  questionnaire,  in  part  because  they  rely  on  another  agency, 
vendor,  or  contractor  to  provide  electronic  access  to  their  products. 
Sometimes  these  "host  disseminators,"  such  as  GPO,  assisted  in  preparing 
the  responses  sent  in  by  the  publishing  entity. 

Observations  about  the  data  collection  process.  Agency  coordinators 
had  difficulty  locating  a  single  point  of  contact  from  each  agency  sub-unit 
who  was  knowledgeable  about  the  range  and  type  of  electronic  information 
products  created  for  the  agency.  Furthermore,  due  to  the  nature  of  the 
survey  questions,  product  respondents  had  to  coordinate  responses  to  some 
questions  with  personnel  who  often  did  not  work  in  their  program  areas. 
This  process  required  respondents  to  identify  personnel  with  whom  they 
appeared  to  have  little  prior  contact,  such  as  records  managers,  information 
technology  staff,  and  staff  in  planning  offices,  in  order  to  respond  to  these 
questions.  In  some  cases,  this  extra  step  discouraged  respondents  from 
seeking  answers  to  these  questions,  so  questions  were  left  unanswered. 
Also,  agencies  whose  coordinators  could  not  attend  the  coordinator 
briefings  and  agencies  that  did  not  participate  in  the  agency  meetings  had 
more  problems  with  data  consistency  than  did  other  agencies. 

The  qualitative  data  collection  included  site  visits,  agency  meetings,  and 
expert  interviews.  Westat  conducted  site  visits  to  Federal  depository 
libraries  from  July  30  through  September  9,  1998.  The  statement  of  work 
(Appendix  J)  specified  that  Westat  visit  three  libraries:  one  regional 
academic,  one  law,  and  one  public.  Furthermore,  GPO  suggested  that 
Westat  visit  the  following  specific  libraries  in  the  Washington,  D.C., 
metropolitan  area: 


Methodology  for 
Qualitative  Data 
Collection 

Site  Visits  to 
Depository  Libraries 
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•  McKeldin  Library,  University  of  Maryland  College  Park,  College  Park, 
Maryland 

•  Washington    College    of    Law    Library,    American  University, 
Washington,  D.C. 

•  Montgomery  County  Rockville  Regional  Public  Library,  Rockville, 
Maryland 

The  purpose  of  visits  was  to  discuss  the  effects  of  the  transition  to  a  more 
electronic  Federal  Depository  Library  Program  on  the  end  user  and  on  the 
services  and  resources  of  each  library.  The  interview  questions,  which  were 
based  on  readings  and  discussions  with  GPO  and  NCLIS,  covered  three 
broad  areas: 

•  What  key  issues  or  concerns  do  you  have  about  users  accessing  and 
using  electronic  Government  information  products? 

•  What   are  your  concerns   about   providing  access  to  electronic 
Government  information  products? 

•  What  specific  ideas  do  you  have  for  improving  public  access  to  online 
and  electronic  Government  information  products  in  your  library? 

The  site  visits  were  audiotaped.  In  addition,  the  libraries  gave  Westat 
representatives  a  tour  of  the  facilities.  (See  Appendix  F  for  a  list  of  the 
specific  interview  questions,  the  names  of  all  interviewees,  and  detailed  site 
visit  notes.) 

Site  visit  observations.  In  addition  to  the  small  number  of  libraries  visited, 
the  problems  and  concerns  of  librarians  in  the  D.C.  metropolitan  area  may 
not  be  representative  of  those  experienced  by  librarians  at  most  depository 
libraries,  especially  the  selective  depositories.  Some  smaller  selective 
depository  libraries  that  are  located  in  more  remote  areas  as  well  as  some  of 
the  larger  urban  selective  depositories  might  have  fewer  resources  (e.g., 
fewer  computers  and  trained  librarians,  training  funds,  and  options  for  low- 
cost  Internet  providers). 


Purpose  and 
Procedures  for 
Agency  Meetings 


Meetings  with  agency  representatives  were  held  between  September  1 5  and 
September  24,  1998.  The  purpose  of  the  meetings  was  twofold: 

•  To  collect  qualitative  data  about  electronic  Government  information 
products  that  were  not  covered  in  the  survey,  such  as  cost-effectiveness 
of  standards,  use  of  locator  tools,  results  of  user  surveys,  etc.;  and 

•  To  discuss  the  questionnaire  and  data  collection  procedures  for 
distribution  of  the  questionnaire. 
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In  addition  to  inviting  agency  coordinators  and  respondents,  the  statement 
of  work  specified  that  Westat  invite  representatives  of  the  following  offices 
to  attend  the  meetings: 

•  Public  affairs  or  communications  offices, 

•  Agency  printing  and  publishing  units, 

•  Information  technology  or  electronic  information  systems  offices, 

•  Agency  libraries;  and 

•  Relevant  program  offices. 

Westat  wrote  the  procedures  for  scheduling  agency  meetings  and  arranging 
for  logistics,  which  included  developing  meeting  protocols,  agenda,  cover 
letter,  and  script  for  interviewers  to  schedule  meetings.  We  then  contacted 
coordinators  and  sent  them  the  following  materials: 

•  Cover  letter  explaining  purpose  of  meeting  and  their  tasks, 

•  Meeting  agenda  and  discussion  questions, 

•  Press  release  from  NCLIS  with  background  information  on  the  project, 

•  Roster  of  potential  agency  representatives  who  will  attend  meeting  (to 
be  completed  by  the  coordinator),  and 

•  Respondent  product  roster  (to  be  completed  by  coordinator). 

Agency  meetings  held.  Westat  contacted  15  of  the  24  agencies  to  hold 
meetings.  Of  the  7  agencies  that  were  not  contacted,  3  had  fewer  than  10 
products.  NCLIS  instructed  Westat  not  to  hold  meetings  with  the  U.S. 
Congress  and  the  Executive  Office  of  the  President  because  NCLIS  and 
GPO  worked  with  them  directly. 

Ten  of  the  16  agencies  did  not  respond  to  Westat' s  request  to  schedule  a 
meeting.  The  following  six  agencies  agreed  to  schedule  a  meeting: 

•  Department  of  Health  and  Human  Services 

•  Department  of  Education 

•  U.S.  Supreme  Court 

•  Department  of  Commerce 

•  Environmental  Protection  Agency 

•  National  Archives  and  Records  Administration 

Only  four  of  the  above  six  agencies  chose  to  discuss  the  qualitative 
questions  at  the  meeting.   The  other  two  agencies  wanted  to  discuss  the 
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questionnaire  only  and  agreed  to  respond  to  the  discussion  questions  in 
writing.  However,  only  one  of  them  sent  in  responses. 

Westat  audiotaped  all  agency  meetings  and  took  notes  as  agency  personnel 
discussed  the  questions.  (Appendix  G  includes  the  list  of  agencies  that 
participated  in  meetings,  the  number  of  attendees,  the  discussion  questions, 
and  summary  notes  from  the  meetings.) 

In  addition  to  vthe  meetings  held  with  Westat,  NCLIS  and/or  GPO 
representatives  met  with  approximately  50  agency  representatives.  In  these 
meetings,  NCLIS  and  GPO  discussed  survey  goals  and  objectives  and  the 
process  for  preselecting  products,  in  addition  to  responding  to  specific 
questions  about  the  survey. 

Agency  meeting  observations.  Agency  participation  in  the  entire  project 
was  voluntary  but  essential.  As  with  any  voluntary  activity,  participation  is 
based  on  availability  and  timing.  For  example,  many  agency  coordinators 
were  unavailable  to  schedule  meetings  during  the  summer  months,  or  they 
were  available  but  product  respondents  were  on  vacation,  which  may  have 
resulted  in  fewer  agency  meetings. 

Product  respondents  needed  to  attend  the  agency  meetings  to  review  the 
questionnaire,  although  they  were  not  always  the  most  appropriate 
personnel  to  respond  to  all  of  the  qualitative  questions.  The  project 
depended  upon  the  good  faith,  interest,  and  cooperation  of  agency  CIOs  and 
coordinators  to  participate  in  the  meetings.  Respondents  and  participants 
from  the  private  sector  are  often  given  an  honorarium  for  participating  in 
similar  research  activities,  but  Federal  employees  are  exempt  from  this 
process. 

Scheduling  agency  meetings,  calling  coordinators,  and  preparing  paperwork 
to  send  to  coordinators  took  a  considerable  amount  of  planning  and 
coordination  and  time,  but  it  did  not  result  in  many  meetings.  Agencies 
were  cooperative,  but  it  was  difficult  for  them  to  identify  the  "right" 
personnel  to  invite  to  the  meetings,  even  though  coordinators  took  a 
significant  amount  of  time  to  locate  product  respondents  from  other  sub- 
units  within  their  agencies.  Therefore,  answers  to  the  agency  meeting 
discussion  questions  reflected  the  perspectives  of  only  5  of  the  24  agencies 
surveyed. 

NCLIS  provided  a  list  of  experts  from  which  Westat  chose  six  names. 
Westat  held  four  telephone  interviews  with  the  six  experts  between  October 
27  and  November  24,  1998.  The  experts  included  two  webmasters,  two 
preservation  specialists,  and  two  professors  in  information  resources 
management.  The  purpose  of  expert  interviews  was  to: 


Expert  Interviews 
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•  Solicit  opinions  of  experts  on  topics  not  adequately  covered  on  the 
survey  or  in  the  agency  meetings, 

•  Ask  questions  raised  during  the  agency  meetings  or  site  visits  that 
require  further  explanation,  or  to  provide  a  broader  context  in  which  to 
view  the  issues,  and 

•  Explore  current  initiatives  and  future  directions. 


As  with  the  site  visits  and  agency  meetings,  Westat  audiotaped  the 
interviews.  Appendix  H  provides  a  list  of  experts,  interview  questions,  and 
a  summary  of  interview  notes. 
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3 


Survey 
Analysis 
and  Findings 


This  section  of  the  report  presents  the  survey  findings  from  each  of  the 
major  survey  questions  as  they  appear  in  the  questionnaire  (Appendix  E). 
Appendix  E  also  includes  a  glossary  of  terms  and  acronyms  used 
throughout  this  report.  The  discussion  and  presentation  will  then  focus  on 
the  key  study  questions  explored  on  the  following  topics: 


Preferred  medium  and  formats  used, 
Planned  medium  and  format  changes, 
Permanent  public  access  issues, 
Permanent  retention  issues, 
Authenticity,  and 

Searchability,  proprietary  software,  and  licensing  fees. 


The  final  response  rate  was  74  percent.  Respondents  from  24  Government 
agencies  completed  and  returned  a  total  of  242  of  the  328  questionnaires 
fielded.  The  word  "respondents"  refers  to  the  242  agency  personnel  who 
completed  the  questionnaire.  Since  each  agency  submitted  at  least  two 
product  questionnaires,  the  unit  of  analysis  is  the  product  or  product 
respondent,  not  the  agency  (table  1 ).  The  sample  was  not  randomly  selected 
due  to  cost  and  time  constraints.  Therefore,  readers  are  cautioned  about 
generalizing  the  findings  to  all  electronic  Government  information  products. 


Structure  of  the  ^he  questionnaire  is  organized  into  five  sections,  A  through  E.  Section  A 

Onpctfnnnairp  contains  general  information  about  the  product  and  agency  that  produced  it. 

Section  B  contains  questions  about  the  product's  current  profile  including 
the  kinds  of  data  the  product  contains,  mediums  in  which  it  is  produced, 
and,  if  in  an  online  medium,  formats  and  online  approaches  used.  This 
section  concludes  with  questions  on  searchability  and  retrievability  of  the 
product.  Section  C  relates  to  the  future  plans  for  the  product  and  is 
designed  to  solicit  information  about  changes  in  the  product's  data, 
mediums,  and  formats.  Section  D  addresses  the  issues  of  metadata, 
permanent  public  access,  permanent  retention,  authenticity, 
updating/upgrading  plans,  user  fees,  licensing,  and  public  domain.  The 
final  section,  E,  includes  one  open-ended  general  comments  question. 
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Section  A  Responses  Sections  A  and  B  of  the  questionnaire  focus  on  format  and  medium 

standards  that  address  the  key  objectives  of  the  study.  Section  A  contains 
general  information  about  the  product  and  the  agency  that  produced  it, 
including  the  name  of  the  agency  and  its  sub-unit,  the  product  name  and 
description,  and  the  Uniform  Resource  Locator  (URL)  for  the  site  in  which 
the  product  appears.  A  list  of  the  agencies  surveyed  and  the  number  of 
product  questionnaires  received  from  each  agency  appears  in  table  1 .  (For 
a  description  of  how  products  were  selected,  refer  to  the  methodology 
section.)  Appendix  C  contains  the  final  list  of  products  surveyed. 


Table  1. 


Number  of 

Agency 

surveys  returned 

Administrative  Office  of  the  U.S.  Courts   5 

Department  of  Agriculture   19 

Department  of  Commerce   14 

Department  of  Defense   8 

Department  of  Education   14 

Department  of  Energy   12 

Department  of  Health  and  Human  Services   19 

Department  of  the  Interior   1 1 

Department  of  Justice   8 

Department  of  Labor   2 

Department  of  State   3 

Department  of  Transportation   9 

Department  of  the  Treasury   13 

Environmental  Protection  Agency   16 

Executive  Office  of  the  President   5 

General  Services  Administration   8 

Library  of  Congress   2 1 

National  Aeronautics  and  Space  Administration   6 

National  Archives  and  Records  Administration   10 

Securities  and  Exchange  Commission   1 1 

Smithsonian  Institution   11 

Social  Security  Administration   4 

Supreme  Court  of  the  United  States   4 

United  States  Congress   9  

SOURCE:  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product 
Assessment  Questionnaire:  1 998 
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Section  B  Responses  Section  B  covers  the  current  product  profile,  including: 

•  How  it  is  used; 

•  What  types  of  data  it  contains; 

•  What  mediums  the  product  is  available  in,  what  is  the  primary  medium 
used,  and  what  are  the  agency's  medium  standards; 

•  What  kinds  of  formats  are  used,  what  is  the  primary  format  used,  and 
what  are  the  agency's  format  standards; 

•  What  user  interfaces  are  supported  and  what  web  design  approaches 
are  used; 

•  If  the  electronic  product  can  be  searched  and  how; 

•  What  agency  hosts  the  product  on  the  web;  and 

•  How  the  product  can  be  retrieved. 

Readers  should  note  that  most  of  the  survey  questions  asked  respondents  to 
"check  all  that  apply";  therefore,  the  percentages  for  these  questions  will 
exceed  100  percent.  Also,  for  the  first  set  of  tables  in  this  section  (tables  1 
through  6),  the  response  categories  appear  in  descending  order  by  number 
or  percentage.  Therefore,  the  responses  will  not  match  the  order  in  which 
they  appear  on  the  questionnaire. 

Types  of  Data  Contained  in  Product 

Table  2a  shows  that  the  frequently  mentioned  types  of  data  contained  in  the 
products  surveyed  are  textual  (188  responses),  followed  by  graphical  (142 
responses),  numerical  (141  responses),  bibliographic  (82  responses),  and 
spatial  (53  responses).  Multimedia,  video,  and  sound  are  less  common, 
probably  because  they  reflect  the  products  surveyed  and  because  of  the 
special  plug-ins,  hardware,  and  memory  required  to  open,  view,  and  listen 
to  products  that  contain  these  data  types.  The  primary  data  types  contained 
in  products  surveyed  are  textual  (57  percent),  numerical 
(21  percent),  bibliographic  (10  percent),  and  graphical  (5  percent; 
table  2b).  These  four  types  of  data  account  for  approximately  93  percent  of 
the  products  surveyed. 
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Table  2a. 

Number  and  percent  of  types  of  data,  by  the  type  of  data  contained 


Type  of  data 


Type  of  data  contained 


Number 


Percent 


Textual  data  (books,  serials,  reports)   188  77.7 

Graphical  data  (photos,  charts,  graphs,  tables,  drawings)   142  58.7 

Numerical  data   141  58.3 

Bibliographic  data  „   82  33.9 

Spatial  data  (maps,  coordinate  files)   53  21.9 

Multimedia  (sound,  video,  text,  graphics)   14  5.8 

Video   10  4.1 

Sound   9  3.7 

Other   16  6.6 


NOTE:  Percents  do  not  add  to  100  because  respondents  could  choose  more  than  one  item. 

SOURCE:  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product 

Assessment  Questionnaire,  1998. 


Table  2b. 

Number  and  percent  of  types  of  data,  by  the  primary  type  of  data 


Type  of  data 


Primary  type  of  data 


Number 


Percent 


Textual  data  (books,  serials,  reports)   138  57.0 

Numerical  data   50  20.7 

Bibliographic  data   24  9.9 

Graphical  data  (photos,  charts,  graphs,  tables,  drawings)   13  5.4 

Multimedia  (sound,  video,  text,  graphics)   3  1.2 

Spatial  data  (maps,  coordinate  files)   2  0.8 

Sound   1  0.4 

Video   1  0.4 

Other    10  4_1_ 

NOTE:  Percents  do  not  add  to  100  because  of  rounding. 


SOURCE:  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product 
Assessment  Questionnaire,  1998. 


Types  of  Mediums  Used 

Respondents  were  asked  to  identify  all  the  types  of  mediums  in  which  the 
product  is  available  to  the  public  as  well  as  the  primary  type  of  medium 
used.  The  most  common  type  of  medium  used  among  pre-electronic 
mediums  is  paper  (177  responses),  followed  by  microform  (22;  table  3a). 
The  responses  in  the  "other"  category  include  Fax  on  Demand,  audiotapes, 
and  Braille.  Among  electronic  mediums  used,  it  is  not  surprising  that  the 
web     is     the     most     common     (204     responses),     followed  by 
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CD-ROM  (70  responses),  floppy  diskettes  (42  responses),  hard  drive 
(30  responses),  and  magnetic  tape  (18  responses).  These  figures  reflect  the 
medium  types  the  public  is  most  likely  to  easily  access,  as  well  as  the 
availability  and  growing  interest  in  the  web. 

Table  3a  also  displays  the  frequency  and  percentage  distribution  of  the 
primary  types  of  mediums  in  which  the  product  is  publicly  accessible.  The 
web  (42  percent)  and  paper  (4 1  percent)  are  the  primary  types  of  mediums 
used,  followed  by  CD-ROM  (8  percent)  as  a  distant  third. 

Standards  for  all  mediums  checked.  For  each  type  of  medium  checked, 
respondents  identified  one  medium  standard  (see  Appendix  E  glossary) 
among  four  types: 

•  Agency  mandated, 

•  Common  agency  practice, 

•  Other,  and 

•  None. 

While  most  agencies  have  some  type  of  standards  for  their  pre-electronic 
and  electronic  mediums,  they  are  primarily  "common  agency  practice" 
rather  than  "agency  mandated."  For  pre-electronic  mediums,  33  percent  of 
the  products  in  paper  are  in  an  agency-mandated  standard  (table  3b). 
However,  52  percent  of  paper  products  are  used  as  a  common  agency 
practice.  Only  13  percent  of  the  CD-ROM  products  are  in  an  agency- 
mandated  standard,  as  compared  to  59  percent  of  CD-ROMs  that  are  used 
as  a  common  agency  practice.  Eighteen  percent  of  web-based  products 
were  reported  to  be  in  an  agency-mandated  standard,  while  70  percent  of 
them  are  used  as  a  common  agency  practice. 

A  considerable  number  of  products  in  CD-ROM  (21  percent)  were  reported 
as  having  no  standards  (table  3b).  Compare  these  numbers  to 
9  percent  (15  products)  of  products  reported  by  respondents  as  having  no 
standards  for  the  use  of  paper,  and  8  percent  (16  products)  reported  as 
having  no  standards  for  the  use  of  the  web. 


21 


Assessment  of  Electronic  Government  Information  Products 


Table  3a. 

Number  and  percent  of  mediums  publicly  available,  by  the  type  of  medium  used  and  primary  medium 


used 


ivicuium 

Type  of  medium  used1 

Primary  medium  used2 

Number 

Percent 

Number 

Percent 

Online  mediums 

Web  

204 

84.3 

102 

42.1 

Bulletin  Board  Systems  .V  

13 

5.4 

1 

0.4 

Gopher  

10 

4.1 

0 

0.0 

Other  

24 

9.9 

3 

1.2 

Pre-electronic  mediums 

Paper  

177 

73.1 

99 

40.9 

Microform  

22 

9.1 

0 

0.0 

Other  

12 

5.0 

2 

0.8 

Optical  mediums 

CD-ROM  

70 

28.9 

20 

8.3 

WORM  (write  once,  read  many  disk)  

1 

0.4 

1 

0.4 

DVD  (digital  video  disk)  

1 

0.4 

0 

0.0 

Other  

1 

0.4 

1 

0.4 

Electronic  mediums 

Floppy  diskette  

42 

17.4 

1 

0.4 

Hard  drive  

30 

12.4 

4 

1.7 

Magnetic  tape  

18 

7.4 

1 

0.4 

Other  

10 

4.1 

1 

0.4 

'Column  percents  do  not  add  to  100  because  respondents  could  choose  more  than  one  item. 

2  Percents  do  not  add  to  100  due  to  nonresponse. 

SOURCE:  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product  Assessment  Questionnaire,  1998. 
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Table  3b. 


Number  and  percent  of  mediums  publicly  available,  by  the  standard  for  each  medium  used 


Medium 

Standard  for  each  medium  used 

Agency  mandated 

Common  agency 
practice 

Other 

None 

Number 

Percent 

Number 

Percent 

Number 

Percent 

Number 

Percent 

Online  mediums 

Web  

36 

17.6 

143 

70.1 

9 

4.4 

16 

7.8 

Bulletin  Board  Systems  

1 

7.7 

10 

76.9 

0 

0.0 

1 

7.7 

Gopher  

0 

0.0 

8 

80.0 

0 

0.0 

2 

20.0 

Other  

4 

16.7 

14 

58.3 

0 

0.0 

6 

25.0 

Pre-electronic  mediums 

Paper  

58 

33.0 

92 

52.3 

9 

5.1 

15 

8.5 

Microform  

6 

26.1 

14 

60.9 

3 

13.0 

0 

0.0 

Other  

2 

16.7 

6 

50.0 

2 

16.7 

2 

16.7 

Optical  mediums 

CD-ROM  

9 

12.9 

41 

58.6 

4 

5.7 

15 

21.4 

WORM  (write  once,  read  many  disk)  

0 

0.0 

0 

0.0 

0 

0.0 

1 

100.0 

DVD  (digital  video  disk)  

0 

0.0 

0 

0.0 

0 

0.0 

1 

100.0 

Other  

0 

0.0 

1 

100.0 

0 

0.0 

0 

0.0 

Electronic  mediums 

Magnetic  tape  

3 

16.7 

12 

66.7 

1 

5.6 

2 

111 

Floppy  diskette  

3 

7.1 

29 

69.0 

5 

11.9 

5 

11.9 

Hard  drive  

2 

6.7 

24 

80.0 

0 

0.0 

4 

13.3 

Other  

1 

100 

5 

50.0 

4 

40.0 

0 

0.0 

NOTE:  Percents  may  not  add  to  100  because  of  rounding. 

SOURCE:  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product  Assessment  Questionnaire,  1998. 


Format  Types  Used 

Databases.  Responses  to  all  formats  used  are  shown  in  table  4a.  Wide 
Area  Information  Server  (WAIS)  is  the  most  common  type  of  database 
identified  (22  responses),  followed  by  Oracle  (17  responses),  and  dBase 
(9  responses).  In  some  cases,  WAIS  is  reported  because  the  products 
surveyed  are  made  available  through  GPO  Access.  The  44  responses  in  the 
"other"  category  reveal  few  multiple  responses  except  for  Microsoft  Access 
that  received  5  "write-in"  responses  in  this  category. 

WAIS  (24  percent)  and  Oracle  (14  percent)  are  the  primary  types  of 
databases  used  (table  4a).  Ninety-one  percent  of  the  respondents  who 
checked  WAIS  as  one  of  the  databases  used  also  indicated  that  the  use  of 
WAIS  is  a  common  agency  practice,  while  only  one  respondent  indicated 
that  WAIS  is  agency  mandated  (table  4b).  However,  only  44  percent  of  the 
respondents  identified  the  use  of  Oracle  as  a  common  agency  practice, 
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but  39  percent  of  respondents  indicated  their  use  of  Oracle  is  agency 
mandated. 

Spreadsheets.  For  spreadsheet  formats  used,  Excel  and  Lotus  1-2-3 
received  33  and  23  responses,  respectively  (table  4a).  When  respondents 
were  asked  to  choose  one  of  the  databases  as  the  primary  type  used, 
59  percent  chose  Excel,  while  only  33  percent  chose  Lotus  1-2-3.  Close  to 
7 1  percent  of  the  respondents  also  identified  the  use  of  Excel  as  a  common 
agency  practice  as  compared  to  3  8  percent  who  indicated  the  use  of  Lotus 
1-2-3  as  a  common  agency  practice  (table  4b). 

Tagged  mark-up.  Hypertext  Markup  Language  (HTML)  is  both  the  most 
commonly  used  tagged  markup  language  (157  responses)  and  the  primary 
type  of  tagged  markup  language  used  (89  percent;  table  4a).  The 
Government  agencies  surveyed  seldom  use  Extensive  Markup  Language 
(XML)  (2  responses),  and  Standard  Generalized  Markup  Language 
(SGML)  (14  responses).  This  is  noteworthy  since  SGML  is  one  of  the  few 
formats  that  NARA  accepts  for  electronic  records. 

Even  given  the  fact  that  HTML  is  the  primary  type  of  tagged  markup 
format  used,  72  percent  of  the  respondents  reported  that  HTML  is  used  as  a 
common  agency  practice,  while  only  13  percent  reported  that  its  use  is 
mandated  by  the  agency  (table  4b).  Sixty  percent  of  the  respondents  who 
use  SGML  for  their  online  products  reported  it  as  a  common  agency 
practice,  while  only  13  percent  reported  that  its  use  is  mandated  by  the 
agency. 

Image  formats.  Portable  Document  Format  (PDF)  is  the  most  common 
image  format  (132  responses)  and  the  primary  type  of  format  used 
(49  percent)  by  the  agencies  surveyed  in  this  study  (table  4a).  The  use  of 
PDF  is  followed  by  GIF  (99  responses),  JPEG  (77  responses),  then  TIFF 
(36  responses)  as  image  formats  used.  Perhaps  PDF  is  the  most  commonly 
used  format  by  the  agencies  surveyed  because  the  Federal  Government 
disseminates  a  wide  range  and  large  number  of  forms  and  documents  that 
must  be  printed  in  the  exact  format  in  which  they  are  created. 

Almost  64  percent  of  respondents  reported  that  PDF  is  a  common  agency 
practice,  while  16  percent  reported  it  is  mandated  by  the  agency 
(table  4b).  While  a  higher  percentage  of  respondents  reported  using  GIF 
(69  percent)  and  JPEG  (7 1  percent)  as  a  common  agency  practice,  PDF  is 
the  most  used  agency-mandated  image  format  (16  percent). 

Audio  formats.  The  number  of  responses  reported  in  this  category  reflects 
the  small  numbers  of  products  surveyed  that  contain  sound  (see  table  2a). 
WAV  (12  responses)  is  the  most  commonly  used  sound  format  followed  by 
AU  (5  responses),  and  AIFF  with  1  response  (table  4a).  WAV  is  also  the 
primary  type  of  audio  format  used  (73  percent). 
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Table  4a. 


Frequency  and  percent  of  formats  used, 
by  the  type  of  format  used  and  primary 
type  of  format  used  


Format 

Type  of  format  used' 

Primary  type  of 
format  used2 

Number 

Percent 

Number 

Percent 

WAIS  

ZZ 

Q  1 

y.  1 

zz 

Oracle  

17 

7.0 

13 

14.4 

dBase  

9 

3.7 

8 

8.9 

4 

1.7 

1 

1.1 

MARC 

2 

0.8 

3 

3.3 

Other 

44 

18.2 

40 

AAA 

Spreadsheet 

Excel 

33 

13.6 

30 

coo 

58.6 

1  ntus  1-2-3 

23 

9.5 

17 

55.5 

Other 

3 

1.2 

2 

3.9 

"TqcjctpH  marknn 

l  aggtu  ill  a  I  r\U 

UTU1 

157 

64.9 

152 

88.9 

SGML  

14 

5.8 

8 

4.7 

XML  

2 

0.8 

0 

0.0 

Other  

12 

5.0 

10 

5.8 

Image 

PDF  

132 

54.5 

84 

48.8 

GIF  

99 

41.1 

43 

25.0 

JPEG  

77 

31.8 

16 

9.3 

TIFF  

36 

14.9 

4 

2.3 

Other  

19 

7.9 

13 

7.6 

Table  4a. 

Frequency  and  percent  of  formats  used, 
by  the  type  of  format  used  and  primary 


type  of  format  used  (continued 

) 

Format 

Type  of  format  used1 

Primary  type  of 
format  used2 

Number 

Percent 

Number 

Percent 

Audio 

WAV  

12 

5.0 

1 1 

73.3 

A  I  1 

5 

2.1 

0 

0.0 

Alrr  

1 

0.4 

0 

0.0 

4 

1.7 

3 

20.0 

Video 

MPEG  

9 

3.7 

4 

26.7 

MOV 

7 

2.9 

5 

33.3 

AVI  

4 

1.7 

4 

26.7 

1 

0.4 

1 

6.7 

Text 

ASCII  

122 

50.4 

106 

80.9 

ANSI  

11 

4.5 

0 

0.0 

RTF  

9 

3.7 

5 

3.8 

Other  

17 

7.0 

17 

13.0 

Word  processing 

Word  Perfect 

75 

31.0 

62 

63.9 

Microsoft  Word 

55 

22.7 

21 

21.6 

Other  

23 

9.5 

10 

10.3 

Other  

5 

2.1 

3 

60.0 

'Column  percents  do  not  add  to  100  because  respondents  could 
choose  more  than  one  item. 


2  Percents  do  not  add  to  100  due  to  nonresponse. 
SOURCE:    National  Commission  on  Libraries  and  Information 
Science,      Government      Information      Product  Assessment 
Questionnaire,  1998. 


Sixty-two  percent  or  eight  of  the  agency  respondents  who  indicated  using 
WAV  reported  it  as  a  common  agency  practice;  only  two  respondents 
(15  percent)  reported  that  WAV  is  an  agency-mandated  standard 
(table  4b).  Perhaps  it  is  not  surprising  that  WAV  is  the  most  commonly 
used  audio  format;  since  it  was  built  into  Windows95,  it  has  become  the  de 
facto  standard  for  sound  on  PCs.  AIFF  is  the  standard  audio  format  for 
Macintosh  computers  (PC  Webopaedia;  see  www.pcwebopaedia.com). 
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Table  4b. 


Number  and  percent  of  formats  used,  by  the  standard  for  each  format  used 


Format 

Standard  for  each  format  used 

Agency  mandated 

Common  agency 
practice 

Other 

None 

Number 

Percent 

Number 

Percent 

Number 

Percent 

Number 

Percent 

Database 

Oracle  

v  7 

38.9 

8 

44.4 

1 

5.6 

1 

5.6 

WAIS  

1 

4.3 

21 

91.3 

0 

0.0 

0 

0.0 

MARC  

1 

33.3 

1 

33.3 

0 

0.0 

0 

0.0 

Sybase  

0 

0.0 

4 

80.0 

0 

0.0 

0 

0.0 

dBase  

0 

0.0 

8 

80.0 

0 

0.0 

0 

0.0 

Other  

2 

4.4 

21 

46.7 

12 

26.7 

9 

20.0 

Spreadsheet 

Lotus  1-2-3  

6 

25.0 

9 

37.5 

3 

12.5 

5 

20.8 

Excel  

4 

11.8 

24 

70.6 

0 

0.0 

4 

11.8 

Other  

0 

0.0 

0 

0.0 

0 

0.0 

2 

50.0 

Tagged  markup 

HTML  

21 

13.3 

114 

72.2 

6 

3.8 

15 

9.5 

SGML  

2 

13.3 

9 

60.0 

2 

13.3 

1 

6.7 

XML  

0 

0.0 

1 

33.3 

0 

0.0 

1 

33.3 

Other  

0 

0.0 

7 

53.8 

1 

7.7 

4 

30.8 

Image 

PDF  

21 

15.9 

84 

63.6 

15 

11.4 

11 

8.3 

GIF  

11 

11.0 

69 

69.0 

11 

11.0 

8 

8.0 

JPEG  

7 

9.1 

55 

71.4 

10 

13.0 

4 

5.2 

TIFF  

2 

5.4 

22 

59.5 

10 

27.0 

2 

5.4 

Other  

3 

13.6 

6 

27.3 

10 

45.5 

2 

9.1 

Video  formats.  As  with  the  audio  formats  used,  the  even  smaller  number 
of  responses  reported  in  this  category  also  reflect  the  small  numbers  of 
products  surveyed  that  contain  moving  images.  Table  4a  shows  that 
Moving  Picture  Experts  Group  (MPEG)  (9  responses)  is  the  most 
commonly  used  format,  followed  by  MOV  (7  responses)  and  Audio  Video 
Interleave  (AVI)  (4  responses).  MPEG  may  be  more  commonly  used  since 
it  generally  produces  better  quality  video  than  AVI  (PC  Webopaedia).  Of 
all  the  video  formats  used,  however,  the  primary  type  of  video  format  used 
is  MOV  (33  percent),  followed  by  MPEG  and  AVI 
(27  percent  each). 
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Table  4b. 

Number  and  percent  of  formats  used,  by  the  standard  for  each  format  used  (continued) 


Format 


Standard  for  each  format  used 


Agency  mandated 

Common  agency 
practice 

Other 

None 

Number 

Percent 

Number 

Percent 

Number 

Percent 

Number 

Percent 

Audio 

WAV   2  15.4  8  61.5  0  0.0  2 

AU   4  66.7  0  0.0  0  0.0  1 

AIFF   1  50.0  0  0.0  0  0.0  0 

Other  .   0  0.0  1  20.0  2  40.0  1 

Video 

MOV   0  0.0  5  62.5  0  0.0  2 

MPEG   1  ...      10.0  5  50.0  1  10.0  2 

AVI   1  20.0  3  60.0  0  0.0  0 

Other   0  0.0  0  0.0  1  50.0  0 

Text 

ASCII   14  11.4  87  70.7  6  4.9  14 

RTF   4  40.0  4  40.0  1  10.0  0 

ANSI   1  8.3  10  83.3  0  0.0  0 

Other   3  16.7  11  61.1  2  11.1  1 

Word  processing 

WordPerfect   19  25.3  45  60.0  1  1.3  8 

Microsoft  Word   8  14.3  33  58.9  12  21.4  2 

Other   1  4.2  6  25.0  12  50.0  3 

Other   2  40.0  1  20.0  1  20.0  0 

NOTE:  Percents  do  not  add  to  100  due  to  nonresponse. 

SOURCE:  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product  Assessment  Questionnaire,  1998. 


15.4 
16.7 
0.0 
20.0 


25.0 
20.0 
0.0 
0.0 


11.4 

0.0 
0.0 
5.6 


10.7 
3.6 
12.5 

0.0 


Of  the  respondents  who  reported  using  MPEG,  50  percent  indicated  its  use 
is  a  common  agency  practice,  while  only  1  respondent  ( 1 0  percent)  reported 
that  its  use  is  agency  mandated.  Sixty-three  percent  of  the  respondents 
reported  that  MOV  is  used  as  a  common  agency  practice,  and  none 
indicated  that  its  use  is  agency  mandated  (table  4b). 

Text  formats.  ASCII  is  by  far  the  most  commonly  used  text  format 
(122  responses)  and  the  primary  type  of  text  format  used  (81  percent;  table 
4a).  The  second  most  commonly  used  text  format  is  ANSI  (11  responses) 
followed  by  Rich  Text  Format  (RTF)  (9  responses).  Seventy-one  percent 
(87)  of  the  respondents  reported  that  their  use  of  ASCII  is  a  common 
agency  practice,  as  compared  to  11  percent  (14)  who  reported  its  use  is 
agency  mandated  (table  4b). 
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Word  processing  formats.  Between  the  two  most  popular  word- 
processing  software  packages,  Microsoft  Word  and  WordPerfect,  the  latter 
(75  responses)  is  more  commonly  used  than  Microsoft  Word 
(55  responses;  table  4a).  These  responses  are  also  consistent  with  the 
primary  type  of  word  processing  used.  Sixty-four  percent  of  respondents 
reported  WordPerfect  as  the  primary  type  of  format  used  while  only  22 
percent  of  respondents  reported  Microsoft  Word  as  the  primary  type  of 
format  used.  PageMaker  received  the  largest  number  of  responses  (5)  in  the 
"other"  category,  v  Nineteen  respondents  (25  percent)  reported  that 
WordPerfect  is  an  agency-mandated  format  standard,  while  only  8 
respondents  (14  percent)  indicated  that  Microsoft  Word  is  an  agency- 
mandated  format  standard  (table  4b). 

Summary  of  format  types  used.  Each  of  the  242  respondents  from  the  24 
agencies  surveyed  was  asked  to  identify  the  primary  type  of  format  used  of 
each  of  the  categories.  The  primary  types  of  formats  used  in  each  category 
are  WAIS,  Excel,  HTML,  PDF,  ASCII,  and  to  a  lesser  degree,  WAV  and 
MOV. 


User  Interfaces 

Online  approaches.  Question  9  on  the  survey  refers  to  online  approaches 
used.  Eighty-five  percent  of  the  respondents  reported  that  their  product  is  in 
an  online  medium  (table  5).  These  respondents  were  then  asked  to  respond 
to  a  set  of  questions  on  user  interfaces  supported  and  web  design 
approaches. 


Table  5. 


Number  and  percent  of  products  reported  as  being  in  an  online  medium 


Product  characteristic 

Number 

Percent 

Is  this  product  in  an  online  medium? 

Yes  

206 

85.1 

No  

33 

13.6 

NOTE:  N  =  242;  3  cases  were  not  ascertained. 

SOURCE:  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product 
Assessment  Questionnaire,  1998. 


User  interface  supported.  Table  6a  shows  that  Netscape  Navigator 
(195  responses)  is  a  more  commonly  supported  browser  than  Internet 
Explorer  (170  responses).  However,  close  to  70  percent  of  agency 
respondents  indicated  that  both  of  these  browsers  are  almost  equally 
supported  as  a  common  agency  practice  rather  than  an  agency-mandated 
standard  (table  6b). 
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Table  6a. 

Number  and  percent  of  online  approaches 


used,  by  type  of  online  tool  used 

Type  of  online 

Online  approach 

tool  used 

Number 

Percent 

User  interfaces  supported 

Netscape                                         195  93.3 

Internet  Explorer                                170  81.7 

FTP                                             40  19.1 

Telnet                                           27  12.9 

Nongraphical/dial-up  shell                      15  7.2 

Other                                           22  10.5 

Web  design  approaches 

Basic  HTML  only                              150  71.8 

Tables                                         Ill  53.1 

CGI  Scripts                                       66  31.6 

Frames                                          53  25.4 

Use  of  Javascript                               43  20  .6 

Use  of  Java  Applets                            23  11.0 

XML                                            11  5.3 

Other                                           25  12.0 

Bulletin  board  systems  (BBS) 

Graphical  interface/browser   7  3_4 

NOTE:  Percents  do  not  add  to  100  because  respondents  could 
choose  more  than  one  item. 

SOURCE:  National  Commission  on  Libraries  and  Information 

Science,  Government  Information  Product  Assessment 
Questionnaire,  1998. 


In  addition,  respondents  reported  that  file  transfer  protocol 
(FTP),  Telnet,  and  nongraphical/dial-up  shells  are  also 
supported  by  their  agencies  (table  6a).  Designs  that 
support  LYNX,  a  text-based  browser,  account  for  1 2  of  the 
22  responses  in  the  "other"  category.  The  number  of 
responses  for  the  category  nongraphical/dial-up  shell  are 
low  (15  responses),  especially  given  the  need  for  agencies 
to  comply  with  the  American  with  Disabilities  Act  by 
making  their  sites  more  accessible  to  the  visually  and 
hearing  impaired. 

Like  the  browsers,  the  other  user  interfaces  supported  are 
primarily  supported  as  a  common  agency  practice  rather 
than  an  agency-mandated  standard.  Almost  83  percent  of 
the  40  respondents  who  reported  their  agency  supports 
FTP  also  reported  it  is  a  common  agency  practice,  while  79 
percent  of  the  27  respondents  who  reported  supporting 
Telnet  also  indicated  it  as  a  common  agency  practice  (table 
6b).  No  respondents  reported  that  Telnet  is  an  agency- 
mandated  standard;  however,  8  percent  reported  that  FTP 
is  an  agency-mandated  standard  for  their  surveyed 
products.  Seventy-five  percent  of  the  respondents  reported 
the  support  of  a  nongraphical/dial-up  shell  as  a  common 
agency  practice  while  only  13  percent  indicated  that  it  is 
agency  mandated. 

Web  design  approaches.  Various  web  design  approaches 
used,  in  descending  order,  are  HTML 
(150  responses),  tables  (111  responses),  CGI  Scripts 
(66  responses),  frames  (53  responses),  Javascript 
(43  responses),  Java  Applets  (23  responses),  and  XML  (1 1 
responses;  table  6a).  ColdFusion  was  reported  in  three  of 
the  responses  in  the  "other"  category. 


The  use  of  these  web  design  approaches  is  overwhelmingly 
a  common  agency  practice  rather  than  an  agency-mandated 
standard  (table  6b).  Basic  HTML — tags  that  consistently 
display  content  in  a  similar  fashion  by  the  most  popular 
browsers— is  the  only  approach  to  which  almost  one-fifth 
(18  percent)  of  the  respondents  reported  that  its  use  is 
agency  mandated.  Less  than 

10  percent  of  the  respondents  using  each  of  the  other 
approaches  indicated  that  they  are  agency-mandated 
standards.  Since  the  use  of  frames,  Javascript,  Java 
Applets,  and  XML  may  not  be  supported  or  enabled  for 
many  users'  browsers,  the  agencies  surveyed  appear  to  be 
adopting  them  slowly,  if  at  all. 
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Table  6b. 

Number  and  percent  of  online  approaches  used,  by  the  standard  for  each  online  tool  used 


Standard  for  each  online  tool  used 


Online  approach 

Agency  mandated 

Common  agency 
practice 

Other 

None 

Number 

Percent 

Number 

Percent 

Number 

Percent 

Number 

Percent 

User  interfaces  supported 

Netscape  

  36 

18.3 

139 

70.6 

4 

2.0 

17 

8.6 

Internet  Explorer  

  25 

14.5 

120 

69.4 

4 

2.3 

22 

12.7 

FTP  

  3 

7.5 

33 

82.5 

0 

0.0 

3 

7.5 

Nongraphical/dial-up  shell  

  2 

12.5 

12 

75.0 

0 

0.0 

1 

6.3 

Telnet  

  0 

0.0 

22 

78.6 

1 

3.6 

4 

14.3 

Other  

1 

A  Q 

1  O 

OJ.  / 

A 
U 

U.U 

1 
1 

A  0 

\Vph  Hpcion  onnrntirhpc 
TT  CU  UCMgll  a  \t  Jil  Udl  lit  S 

Basic  HTML  only  

  27 

17.9 

105 

69.5 

1 

0.7 

16 

10.6 

Tables  

  10 

8.9 

88 

78.6 

3 

2.7 

9 

8.0 

CGI  Scripts  

  4 

6.0 

55 

82.1 

0 

0.0 

7 

10.4 

Use  of  Javascript  

  3 

6.8 

30 

68.2 

6 

13.6 

4 

9.1 

Frames  

  2 

3.7 

44 

81.5 

3 

5.6 

4 

7.4 

Use  of  Java  Applets  

  2 

8.3 

15 

62.5 

3 

12.5 

3 

12.5 

XML  

  0 

0.0 

10 

83.3 

1 

8.3 

0 

0.0 

Other  

  3 

11.1 

17 

63.0 

2 

7.4 

4 

14.8 

Bulletin  board  systems  (BBS) 

Graphical  interface/browser  

  1 

12.5 

5 

62.5 

0 

0.0 

1 

12.5 

NOTE:  Percents  do  not  add  to  100  due  to  nonresponse. 


SOURCE:  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product  Assessment  Questionnaire,  1998. 


Searchability  of  Product 

Searchability  of  an  electronic  product  is  important  for  users  because  it 
allows  them  to  effectively  access  the  information  they  need.  Most  electronic 
products  are  searchable  either  by  full-text  with  no  fielding 
(74  responses)  and/or  by  full-text  and  field  (99  responses;  table  7).  The 
"view  only"  category  contains  a  higher  number  of  responses  than  expected 
(79  responses). 

The  "other"  category  contains  the  following  common  responses: 

•  Inapplicable  because  product  is  in  a  paper  medium  (most  common 
response); 

•  In  PDF,  which  is  not  searchable;  and 

•  Product  is  indexed  by  field  only. 
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Table  7. 

Number  and  percent  of  responses  regarding 
searchability  of  the  product 

Product  characteristic 

Number 

Percent 

Indexed  by  full-text  and  field   99  40.9 

Available  as  "view  only"  —  non-searchable   79  32.6 

Included  as  part  of  a  full-text  searchable  database  with  no       74  30.6 

fielding  

Other   32  13.2 

NOTE:  Percents  do  not  add  to  100  because  respondents  could  choose  more  than  one  item. 

SOURCE:  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product 

Assessment  Questionnaire,  1998. 


Product  "Host" 

Most  of  the  products  surveyed  (199  responses)  were  hosted  by  the  agency 
that  created  them,  although  other  agencies  or  institutions  might  also  host  the 
products  since  respondents  were  asked  to  "check  all  that  apply"  for  this 
question  (table  8).  There  are  fewer  responses  for  products  hosted  by  another 
agency  (42  responses),  a  contractor  (17  responses),  and  an  educational 
institution  (9  responses). 


Table  8. 

Number  and  percent  of  responses  regarding 
where  the  product  is  hosted  


Product  host 


Number 


Percent 


Your  agency   199  82.2 

Another  agency   42  17.4 

Contractor   17  7.0 

Educational  institution   9  3.7 

Other   15  6.2 

NOTE:  Percents  do  not  add  to  100  because  respondents  could  choose  more  than  one  item. 

SOURCE:  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product 

Assessment  Questionnaire,  1998. 
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Retrievability  of  Product 

In  order  to  ensure  broad  access  to  the  product,  the  public  should  be  able  to 
download  and  save  electronic  Government  information  products  without 
restrictions  (GPO,  1996,  p.  7).  Responses  to  Question  1 1  indicate  that  for 
the  most  part,  products  surveyed  for  this  study  can  be  downloaded  and 
saved  without  restrictions  (173  responses;  table  9).  Responses  in  the 
second  category  indicate  that  some  products  cannot  be  downloaded  or  saved 
(20  responses).  A  small  number  of  products  (14)  cannot  be  downloaded  or 
saved  because  their  use  requires  proprietary  software  that  is  not  freely 
distributed  (table  9).  Common  write-in  responses  in  the  "other"  category 
include: 

•  Can  be  downloaded  and  saved,  but  subject  to  restrictions. 

•  Can  be  printed  from  browser,  but  not  downloaded. 

•  Product  available  only  in  paper. 

The  United  States  Advisory  Council  on  the  National  Information 
Infrastructure,  in  its  publication  "A  Nation  of  Opportunity,"  identifies  as 
one  of  the  basic  principles  of  Government  information  and  services  that  "the 
Federal  Government  should  not  charge  for  making  its  information 
available... nor  charge  for  access  to  that  information"  (GPO,  1996,  p.  28). 


Table  9. 

Number  and  percent  of  responses  concerning 

the  retrievability  status  of  the  product 

Product  and  software  characteristic 

Number 

Percent 

Can  be  downloaded,  saved,  and  is  not  subject  to  any 

restrictions  on  use  or  re-use  by  the  end  user   173 

Cannot  be  downloaded,  saved,  and/or  re-used  because  it  is 
part  of  a  database  and  does  not  exist  as  a  distinct 
product   20 

Cannot  be  downloaded,  saved,  and/or  re-used  because  it 
requires  proprietary  software  that  is  not  freely 
distributable   14 

Other   41 

NOTE:  Percents  do  not  add  to  100  because  respondents  could  choose  more  than  one  item. 

SOURCE:  National  Commission  on  Libraries  and  Information  Science, 

Government  Information  Product  Assessment  Questionnaire,  1998. 


71.5 

8.3 

5.8 
16.9 
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Section  C  Responses  Section  C  contains  a  series  of  questions  related  to  the  future  product  profile. 

(Planned  Product  Profile)      Respondents  were  asked  questions  about  changes  in  the  types  of  data, 

mediums,  and  formats  used  and  reported  on  in  Section  B  of  the 
questionnaire.  Respondents  also  were  asked  to  identify  the  time  span  in 
which  the  changes  would  occur  and  to  describe  the  planned  changes. 


Types  of  Data 

The  first  question  in  this  section  of  the  questionnaire  asked  respondents 
about  plans  to  discontinue  publication  of  the  product.  Only  5  percent  (12) 
of  the  respondents  planned  to  discontinue  the  product  (table  10).  Several  of 
the  most  commonly  listed  responses  provided  for  discontinuation  of  a 
product  was  that  the  product  was  a  one-time  "prototype"  or  that  the  paper 
version  of  the  product  would  be  discontinued. 


Table  10. 

Number  and  percent  of  respondents  reporting  plans  to 

discontinue  publication  of  the  product 

Product  characteristic 

Number 

Percent 

Are  there  any  plans  to  discontinue 

publication  of  this  product? 

Yes  

12 

5.0 

No  

230 

95.0 

SOURCE:  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product 
Assessment  Questionnaire,  1998. 


Table  1 1  shows  responses  to  question  1 3  about  the  kinds  of  data 
(i.e.,  bibliographic,  textual,  graphical)  the  product  will  contain  in  the  future. 
The  majority  of  respondents  (76  percent)  reported  that  the  agency  plans  no 
changes  to  the  product.  Twenty-one  percent  reported  that  the  agency  would 
add  one  or  more  new  types  of  data.  A  total  of  3  percent  reported  either  the 
discontinuation  of  one  type  of  data  (0.4  percent),  or  a  complete  change  to 
new  data  types  (2.6  percent).  Several  respondents  reported  that  the  changes 
in  data  types  would  include  adding  audio  or  video  and  multimedia. 
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Table  11. 

Number  and  percent  of  responses  regarding  the  planned  changes  to  the 

type  of  data  contained  in  the  product 

Product  characteristic 

Number 

Percent 

What  kind  of  data  will  this  product  contain? 

Retain  existing  type(s)  of  data,  no  changes  planned 

178 

76.4 

Retain  existing  type(s)  of  data  and  add  items  of  one  or 

more  new  types  of  data  

48 

20.6 

Discontinue  one  or  more  types  

1 

0.4 

Change  to  new  type(s)  of  data  

6 

2.6 

NOTE:  N  =  242;  9  cases  are  missing. 

SOURCE:  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product 
Assessment  Questionnaire,  1998. 


Most  agency  respondents  reported  that  these  changes  in  data  types  would 
mainly  occur  in  the  short  term  (40  responses)  and,  to  a  lesser  degree,  in  the 
medium  term  (24  responses;  table  12).  Most  respondents  skipped  the 
question  about  long-term  plans  for  changing  data  types.  Respondents  noted 
in  the  "please  specify"  categories  in  questions  13c  and  13e  indicate  that 
respondents'  plans  for  product  changes  have  not  yet  been  solidified. 


Table  12. 

Number  and  percent  of  responses  regarding  the  timeframe  for  planned 

changes  to  the  type  of  data  contained  in  the  product 

Product  characteristic 

Number 

Percent 

Change(s)  will  occur  in  the: 

Short  term:  within  1  year  or  less  

40 

72.7 

Medium  term:  within  2  to  5  years  

24 

43.6 

NOTE:  Percents  do  not  add  to  100  because  respondents  could  choose  more  than  one  item.  Numbers 

reflect  respondents  who  indicated  changes. 

SOURCE:  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product 

Assessment  Questionnaire,  1998. 

Types  of  Mediums 

Responses  to  changes  in  types  of  mediums  parallel  those  for  changes  in  data 
types.  Seventy-six  percent  of  the  respondents  reported  no  plans  to  change 
mediums  (table  13).  Eighteen  percent  of  respondents  reported  that  they  are 
planning  to  add  one  or  more  mediums,  2  percent  indicated  they  will 
discontinue  one  or  more  mediums,  and  3  percent  reported  they  will  change 
to  a  new  type  of  medium. 
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Table  13. 

Number  and  percent  of  responses  regarding  the  planned 

changes  to  the  mediums  used  for  the  future 

Product  characteristic 

Number 

Percent 

What  kind  ot  meaiurn(s)  will  this  product  use:  11 

product  is  delivered  in  more  than  one  medium,  respond 

for  all  mediums. 

Retain  existing  medium(s),  no  changes  planned  

178 

if.  1 

/O.I 

Retain  existing  medium(s)  and  add  items  of  one  or 

more  new  types  of  mi 

42 

17.9 

Discontinue  one  or  more  types  

4 

1.7 

Change  to  new  type(s)  of  medium  

8 

3.4 

No  agency  mandated  medium  applies  

0 

0.0 

NOTE:  N  =  242;  8  cases  are  missing;  2  cases  were  not  ascertained. 

SOURCE:  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product 
Assessment  Questionnaire,  1998. 


The  two  most  frequently  mentioned  additions  to  medium  types  are  to 
provide  web  access  to  the  product,  and  to  make  the  product  available  on 
CD-ROM.  Most  of  the  respondents  (35)  who  reported  changes  in  medium 
types  indicated  that  the  changes  will  occur  in  the  medium  term; 
21  respondents  indicated  that  these  changes  will  occur  in  the  short  term 
(table  14).  Again,  most  respondents  skipped  the  question  about  long-term 
plans  for  changing  product  mediums.  The  few  respondents  who  provided 
descriptions  of  their  long-term  plans  mentioned  that  they  will  produce  the 
product  in  multiple  mediums  (paper  and  web),  or  that  paper  items  will  be 
migrated  to  the  web.  Other  respondents  indicated  that  their  long-term  plans 
are  undetermined  or  undefined. 


Table  14. 


Number  and  percent  of  responses  regarding  the  timeframe  for  planned 
changes  to  product  medium  used  


Product  characteristic 

Number 

Percent 

Change(s)  will  occur  in  the: 

Short  term:  within  1  year  or  less  

21 

37.5 

Medium  term:  within  2  to  5  years  

35 

62.5 

NOTE:  Numbers  reflect  respondents  who  indicated  changes. 

SOURCE:  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product 
Assessment  Questionnaire,  1998. 
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Types  of  Formats 

One  might  expect  to  see  more  dramatic  changes  in  types  of  formats  since 
the  range  of  formats  is  varied  and  broad  (i.e.,  database,  spreadsheet,  tagged 
markup,  image,  etc.).  The  pattern  of  responses  to  question  15  mirrors  the 
responses  to  changes  in  types  of  data  and  mediums,  except  for  the  change  to 
new  types.  Seventy-two  percent  of  the  respondents  reported  no  changes  in 
format  types.  Eighteen  percent  indicated  that  they  are  planning  to  add  one 
or  more  formats,  while  9  percent  reported  they  will  change  to  new  format 
types  (table  15).  This  change  to  new  format  types  is  the  largest  percentage 
change  in  this  category  as  compared  to  changes  to  new  types  of  data  (3 
percent;  table  11)  and  new  types  of  mediums  (also 
3  percent;  table  13).  Respondents  who  provided  specifics  about  the 
changes  to  new  format  types  indicated  these  new  types  would  be  PDF  and 
XML. 


Table  15. 

Number  and  percent  of  responses  regarding  the  planned 
changes  to  the  formats  the  product  will  contain 


Product  characteristic 


Number 


Percent 


What  kind  of  format(s)  will  this  product  contain? 

Same  as  existing  format(s),  no  changes  planned   167  71.7 

Retain  existing  format(s)  and  add  one  or  more  new  format 

types   42  18.0 

Change  to  new  format  types   21  9.0 

Discontinue  one  or  more  types   1  0.4 

No  agency-mandated  format  applies   1  0.4 

NOTE:  N  =  242;  9  cases  are  missing;  1  case  was  not  ascertained. 

SOURCE:  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product 
Assessment  Questionnaire,  1998. 


The  majority  of  respondents  who  reported  changes  indicated  that  they  will 
occur  in  the  short  term  (36  responses),  and/or  the  medium  term 
(32  responses;  table  16).  The  majority  of  respondents  did  not  answer  the 
question  about  long-term  plans  for  changing  formats. 


Table  16. 

Number  and  percent  of  responses  regarding  the  timeframe 

for  planned  changes  to  the  product  format  used 

Product  characteristic 

Number 

Percent 

Change(s)  will  occur  in  the: 

Short  term:  within  1  year  or  less  

36 

55.4 

Medium  term:  within  2  to  5  years  

32 

49.2 

NOTE:  Total  frequency  reflects  respondents  reporting  changes  only.  Percents  do  not  add  to  100  because 

respondents  could  choose  more  than  one  item. 

SOURCE:  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product 

Assessment  Questionnaire,  1998. 
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Section  D  Responses  Section  D  of  the  questionnaire  contains  a  variety  of  questions  in  an  effort  to 

(Other  Information)  answer  some  of  the  critical  issues  of  public  access  to  electronic  information 

products: 

•  Metadata, 

•  Permanent  public  access  (i.e.,  provided  by  what  agency  and  how), 

•  Permanent  retention, 

•  Ensuring  authenticity, 

•  Updating/upgrading  plans, 

•  User  fees, 

•  Licensing,  and 

•  Public  domain. 

Metadata 

Metadata,  data  about  data,  are  important  for  public  access.  Metadata 
refers  to  describing  the  content  of  a  document  or  record  allowing  users  to 
find  Government  information  more  effectively.  Examples  of  metadata 
include  Government  Information  Locator  Service  (GILS)  and  machine- 
readable  cataloging  (MARC)  records.  To  that  end,  one  of  the  requirements 
of  the  Government  Printing  Office  Electronic  Information  Access 
Enhancement  Act  of  1993  (Public  Law  103-40)  was  that  the  Superintendent 
of  Documents  maintain  an  electronic  directory  of  Federal  electronic 
information  (44  U.S.C.,  Section  4101). 

Only  27  percent  of  agency  respondents  reported  that  their  products  have  a 
metadata  record,  while  69  percent  reported  no  metadata  record  exists  for 
their  products  (table  17).  In  the  followup  question,  most  respondents 
identified  their  metadata  records  as  either  MARC  or  GILS.  Another 
5  percent  indicated  they  do  not  know  if  a  metadata  record  exists. 


Table  17. 

Number  and  percent  of  respondents  reporting 
a  metadata  record  for  the  product 

Product  characteristic 

Number 

Percent 

Is  there  a  metadata  record  for  this  product 
(e.g.,  GILS,  MARC)? 

Yes  

65 

26.9 

No  

166 

68.6 

Don't  know  

11 

4.5 

SOURCE:  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product 
Assessment  Questionnaire,  1998. 
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Permanent  Public  Access 

In  an  electronic  age,  permanent  public  access  to  Government  information,  a 
critical  concept  in  information  resources  management,  presents  far-reaching 
challenges  to  the  Federal  Depository  Library  Program,  Congress,  Federal 
agencies,  and  ultimately  the  American  public.  GPO  indicates  that 
permanent  public  access  "means  that  electronic  Government  information 
products  within  the  scope  of  the  FDLP  remain  available  for  continuous,  no- 
fee  public  access  through  the  program"  (GPO,  1998,  p.  19).  GPO 
recognizes  and  acknowledges  its  responsibility  to  provide  ongoing  public 
access  to  the  electronic  Government  information  available  through  the 
FDLP.  However,  in  a  decentralized  networked  environment,  agencies  are 
asked  to  share  the  responsibility  for  building,  storing,  disseminating,  and 
preserving  a  broad  range  of  electronic  information  products  in  order  to 
ensure  continued  public  access. 

Agency  respondents  reported  that  permanent  public  access  is  primarily 
provided  by  their  agency  (177  responses),  by  another  agency 
(51  responses),  and/or  by  some  other  entity  (20  responses;  table  18). 
Respondents  reported  that  permanent  public  access  is  not  provided  for 
28  products  (table  18).  However,  on  closer  examination,  the  responses  to 
the  "please  specify"  questions  indicate  that  either  respondents  may  have 
misunderstood  the  concept  of  permanent  public  access  (as  opposed  to 
current  access),  or  they  assumed  other  entities  have  this  responsibility. 
Some  of  the  common  responses  to  "other"  agencies  include  the  Government 
Printing  Office,  National  Archives  and  Records  Administration  (NARA), 
and  contractors  and  vendors.  These  responses  illustrate  respondents'  lack 
of  understanding  about  the  difference  between  permanent  public  access  to 
electronic  information  products  through  their  own  agencies  or  through 
partnerships  with  GPO,  and  permanent  retention  of  official  Government 
records  through  NARA.  Furthermore,  only  4  of  the  28  products  for  which 
no  permanent  public  access  currently  is  provided  have  future  plans  for 
providing  permanent  public  access  (table  19). 
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Table  18. 

Number  and  percent  of  responses  regarding  the 

entity  providing  permanent  access  to  the  product 

Product  characteristic 

Number 

Percent 

Permanent  public  access  to  this 
product  is  currently  provided  by: 

Your  agency   177  73.1 

Another  agency   51  21.1 

Other   20  8.3 

No  permanent  public  access  provided  ,   28  58.3 

NOTE:  Percents  do  not  add  to  100  because  respondents  could  choose  more  than  one  item. 

SOURCE:  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product 

Assessment  Questionnaire,  1998. 


Table  19. 

Number  and  percent  of  responses  regarding  products 
for  which  access  will  be  provided  in  the  future 


Product  characteristic 


Number 


Percent 


Are  there  plans  to  provide  permanent  public  access  in 
the  future  for  this  product? 

Yes  

No  


4 
24 


12.9 
80.6 


NOTE:  Only  those  respondents  (N=28)  who  indicated  there  is  currently  no  permanent  public  access 
provided  for  this  product  were  asked  to  complete  this  question. 

SOURCE:  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product 
Assessment  Questionnaire,  1998. 


Permanent  Retention 


The  mission  of  the  National  Archives  and  Records  Administration  is 
distinct  from  that  of  GPO.  NARA's  mission  is  to  preserve  and  provide 
public  access  to  permanently  valuable  records  of  the  Federal  Government. 
Federal  agencies  are  responsible  for  transferring  products  to  NARA  that  are 
scheduled  as  permanent  records  (i.e.,  official  records  of  the  Federal 
Government  as  defined  by  the  Federal  Records  Act).  Under  36  CFR 
1228.188,  mediums  approved  for  transfer  include  open  reel  magnetic  tape, 
magnetic  tape  cartridge,  and  CD-ROM.  Agencies  currently  may  not 
transfer  to  NARA  electronic  records  that  are  in  a  format  dependent  on 
specific  hardware  and  software.  However,  SGML  tags  are  permitted  on 
electronic  textual  documents  as  are  records  written  in  ASCII  or  Extended 
Binary  Coded  Decimal  Interchange  Code  (EBCDIC)  with  all  control 
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characters  and  other  non-data  characters  removed  (Lewis  Bellardo,  Deputy 
Archivist  of  the  U.S.  in  a  written  response  to  agency  questions,  October  14, 
1998). 

The  responses  to  the  questions  on  permanent  retention  may  reflect  the 
current  status  of  transferring  permanent  electronic  records  to  NARA  (see 
questions  findings).  Only  34  percent  of  agency  respondents  reported  that 
their  products  are  scheduled  for  permanent  retention  by  NARA  (table  20). 
Sixty- four  percent  reported  their  products  are  not  scheduled  for  retention, 
while  another  3  percent  reported  they  do  not  know  if  the  product  is 
scheduled  for  retention.  However,  it  should  be  pointed  out  that  at  the  time 
of  the  survey,  the  schedule  that  would  have  covered  electronic  records  of 
permanent  value  was  unenforceable  under  a  court  case  declaring  it  null  and 
void;  therefore,  these  figures  may  be  unreliable. 


Table  20. 

Number  and  percent  of  responses  regarding 
permanent  retention  of  the  product  


Product  characteristic 


Number 


Percent 


Is  this  product  scheduled  for  permanent  retention  by  the 
National  Archives  and  Records  Administration? 

Yes   82  33.9 

No   154  63.6 

Don't  know   6  

SOURCE:  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product 
Assessment  Questionnaire,  1998. 


Ensuring  Authenticity 

Although  64  percent  of  respondents  reported  that  their  agency  ensures 
authenticity  for  the  products  surveyed  (table  21),  responses  to  the  open- 
ended  question  about  how  the  agency  attests  to  authenticity  indicate  that 
respondents  may  not  fully  understand  the  concept. 

Authentication  refers  to  the  process  agencies  use  to  ensure  the  public  that 
the  product  is  an  official  legitimate  product  created  and  produced  by  the 
Federal  Government  agency  and  no  other  source  (see  glossary,  p.  E-17). 
Ensuring  authentication  includes  technical  as  well  as  policy  considerations. 
Some  technical  examples  of  authentication  include  digital  signature 
technology,  special  watermarks,  disclaimers,  or  statements  on  the  products. 
Respondents  provided  answers  that  address  how  the  agency  ensures  that 
information   or   data    in   the   product   are   valid   or  reliable —  an 
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important  process,  but  not  the  same  concept  as  authenticity.  Common 
responses  include  the  following: 

•  Program  office  verifies  data. 

•  Review  CD-ROM  contents  before  public  release. 

•  Regulations  and  source/reliability  statement  regarding  data  sources. 

•  Review  and  approval  within  agency. 

•  Source  of  content  is  the  same  as  the  hardcopy  version. 

•  Test  reliability  of  data  every  5  years,  or  more  often. 

•  Publications  are  subjected  to  review  by  subject  matter  expert  and  peer 
review. 


Table  21. 

Number  and  percent  of  respondents  who  reported 
the  agency  ensures  authenticity  for  the  product 

Product  characteristic 

Frequency 

Percent 

Does  the  agency  ensure  authenticity 

(official  status  determination)  for  this  product? 

Yes  

154 

63.6 

No  

86 

35.5 

Don't  know  

2 

0.8 

NOTE:  Percents  may  not  add  to  100  because  of  rounding. 

SOURCE:  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product 
Assessment  Questionnaire,  1998. 


Updating/Refreshing  Plans 

Twenty  percent  of  respondents  reported  that  their  products  are  updated 
annually,  followed  by  daily  (16  percent),  monthly  (12  percent),  and  weekly 
(5  percent;  table  22).  However,  the  majority  (47  percent)  of  respondents 
checked  the  "other"  response  category.  The  write-in  responses  covered  a 
broad  range  of  time  periods  in  which  products  are  updated.  Below  is  a 
sampling  of  multiple  responses: 

•  Quarterly, 

•  As  needed, 

•  Irregularly, 
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•  Not  updated, 

•  Semi-annually, 

•  Every  2  years, 

•  Periodically,  and 


Twice  a  month  with  old  version  staying  on  line. 


Table  22. 

Number  and  percent  of  responses  regarding 

how  frequently  the  product  is  updated  or  refreshed 


Product  characteristic 


Number 


Percent 


How  frequently  is  this  product  updated  or  refreshed? 


Annually  

  48 

19.8 

Daily  

  39 

16.1 

Monthly  

  30 

12.4 

Weekly  

  12 

5.0 

Other  

  113 

46.7 

SOURCE:  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product 
Assessment  Questionnaire,  1998 


Changing  Supporting  Technology 

The  majority  of  the  respondents  (71  percent)  reported  that  there  are  no 
plans  to  change  the  product's  supporting  technology  (table  23).  Twenty- 
eight  percent  of  respondents  reported  plans  to  change  the  product's 
supporting  technology. 


Table  23. 

Number  and  percent  of  responses  regarding 

the  plans  for  supporting  technology  of  the  product 

Product  characteristic 

Number 

Percent 

Are  there  plans  for  changing  the  product's  supporting 
technology? 

Yes   68  28.1 

No   171  70.7 

Don't  know   2  0.8 

NOTE:  N  =  242;  1  case  not  ascertained. 

SOURCE:  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product 
Assessment  Questionnaire,  1998. 
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User  Fees 

Public  access  to  no-fee  Government  information  products  is  one  of  the  core 
principles  upon  which  the  FDLP  is  based.  However,  users  might  be 
charged  a  fee  if  they  order  certain  types  of  electronic  Government 
information  products  directly  from  GPO  or  the  agency  that  created  the 
product.  Nine  percent  of  respondents  reported  that  all  users  are  charged 
fees,  while  20  percent  reported  some  users  are  charged  fees.  The  majority 
(72  percent)  of  agency  respondents  reported  that  there  are  no  fees  charged 
to  access  or  use  the  product  surveyed  (table  24).  The  followup  question 
asks  about  specific  fee  amounts  and  the  reasons  for  the  charge.  The 
responses  to  this  question  vary  greatly.  A  few  common  responses  include 
the  following: 

•  No  charge  for  web  access. 

•  Single  paper  copy  free;  charge  for  additional  copies. 

•  No  subscription  fee  to  libraries  and  some  constituencies. 

•  Files  can  be  downloaded  from  the  Internet  for  free.  There  is  a  charge 
for  published  books. 

•  Fees  are  for  paper  products  only. 


Table  24. 

Number  and  percent  of  respondents  reporting 

that  user  fees  are  charged  for  the  product 

Product  characteristic 

Number 

Percent 

Is  a  user  fee  charged  for  this  product? 

Yes,  for  some  users  

48 

19.8 

Yes,  for  all  users  

21 

8.7 

No  

173 

71.5 

SOURCE:  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product 

Assessment  Questionnaire,  1998. 

Licensing 

Many  Government  agencies  purchase  licenses  from  vendors  for  search  and 
retrieval  software  to  be  used  with  the  product  to  make  the  data  or 
information  more  accessible  to  users.  Agencies  negotiate  various 
agreements  with  vendors  about  who  can  use  the  software  free  of  charge. 
The  majority  of  respondents  (69  percent)  reported  that  they  do  not  license 
commercial  search  and  retrieval  software  (table  25).  For  the  remaining 
31  percent  of  respondents  who  have  licensed  commercial  software,  the 
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license  covers  use  by  all  the  key  constituencies  including  agency  personnel 
(73  responses),  public  users  (69  responses),  agency's  primary  target 
constituencies  (65  responses),  Federal  depository  libraries 
(59  responses),  and/or  all  libraries  (59  responses;  table  26). 


Table  25. 

Number  and  percent  of  respondents  reporting  about  the  use  of  licensed 
commercial  search  and  retrieval  software  for  the  product 


Product  characteristic 


Number 


Percent 


Has  the  agency  licensed  commercial  search  and  retrieval 
software  for  use  with  this  product? 

Yes  

No  


76 
166 


31.4 
68.6 


SOURCE:  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product 
Assessment  Questionnaire,  1998. 


Table  26. 

Number  and  percent  of  responses  regarding  coverage 


Product  characteristic 

Number 

Percent 

Does  the  agency's  license  cover  use  by: 

Agency  personnel  

73 

96.1 

Agency's  primary  target  constituencies  

65 

85.5 

Federal  depository  libraries  

59 

77.6 

All  libraries  

59 

77.6 

Public  users  

69 

90.8 

Other  

10 

13.2 

NOTE:  Percents  do  not  add  to  100  because  respondents  could  choose  more  than  one  item. 

SOURCE:  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product 

Assessment  Questionnaire,  1998. 


Public  Domain 


Public  domain,  a  critical  component  of  public  access,  means  that  the 
information,  product,  or  publication  is  not  copyrighted  and  therefore  can  be 
reproduced  by  anyone  without  obtaining  copyright  permission.  One  of  the 
goals  of  an  electronic  FDLP  is  to  provide  public  access  to  any  Government 
information  product  free  of  copyright  or  copyright-like  restrictions  (GPO, 
1996,  p.  2).  The  majority  of  respondents,  86  percent,  indicated  that  all 
parts  of  their  surveyed  products  are  in  the  public  domain  (table  27). 
Another  10  percent  indicated  that  part  of  the  product  is  in  the  public 
domain,  while  5   percent  reported  that  the  product  is  not  in  the 


44 


Assessment  of  Electronic  Government  Information  Products 


public  domain.  The  followup  question  that  requests  an  explanation  of  the 
second  response  (i.e.,  part  of  product  is  in  the  public  domain)  uncovered 
these  typical  responses: 

•  Copyrighted  tables  are  not  in  the  public  domain. 

•  There  are  some  copyright-protected  logos  and  trademarks. 

•  Includes   copyrighted   material  that   would   require  approval  for 
reproduction. 

Respondents  offered  a  wide  variety  of  explanations  for  products  that  are  not 
in  the  public  domain: 

•  Retrieval  software  is  proprietary  and  use  is  licensed. 

•  Commercial  vendors  lease  the  database  for  distribution. 

•  Songs  and  performances  are  protected  by  copyright. 

•  Books  are  available  only  to  eligible  blind  patrons  of  our  program,  by 
law. 


Table  27. 

Number  and  percent  of  respondents  reporting 

the  public  domain  status  of  the  product 

Product  characteristic 

Number 

Percent 

Is  this  product  in  the  public  domain? 

Yes,  for  the  entire  product  

206 

85.5 

Yes,  for  part  of  the  product  

24 

10.0 

No  

11 

4.6 

NOTE:  N  =  242;  1  case  missing.  Percents  may  not  add  to  100  because  of  rounding. 

SOURCE:  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product 
Assessment  Questionnaire,  1998. 


Section  E  Responses  ^e  ^"ina^  secti°n  E  °f  the  questionnaire  contains  one  open-ended 

"comments"  question.  These  responses  are  too  broad  and  disparate  to 
provide  a  detailed  itemization.  Most  of  the  comments  are  explanations  of 
issues  covered  in  the  survey.  However,  below  are  a  few  comments  that 
cover  issues  not  directly  addressed  in  the  survey. 

•  Our  mission,  mandated  by  the  Americans  with  Disabilities  Act  (ADA), 
is  to  satisfy  all  browser  requirements  (e.g.,  ASCII  browsers  like  LYNX 
through  the  latest  versions  of  Netscape  and  Internet  Explorer). 
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•  We  produce  printed  documents  and  link  to  electronic  documents 
maintained  on  the  GPO's  server. 

•  The  product  is  not  published  in  any  electronic  form.  It  is  a  collection  of 
individual  products  that  are  individually  published. 

•  I  am  very  new  to  this  area  (2  weeks)  and  received  significant  contractor 
assistance  in  completing  this  form. 

•  In  addition  to  four  other  web  sites,  we  will  soon  web-enable  our 
database  with  some  encrypted  modules. 

•  The  database  is  intended  to  be  accessible  to  the  largest  audience 
possible  via  free  or  public  domain  software  whenever  possible. 

•  This  information  is  available  in  PDF  format  on  our  website  to  ensure 
the  integrity  of  the  data.  Coding  in  HTML  (particularly  tables)  could 
lead  to  mistakes  with  such  a  large  amount  of  numeric  data. 

No  respondents  commented  on  the  survey  questionnaire,  the  project  in 
general,  or  the  process  of  filling  out  the  survey. 


Study  Questions  ^ms  sect'on       use  findings  from  two  or  more  survey  questions  to  provide 

additional  information  on  some  of  the  key  issues  explored  in  the  study.  The 
responses  to  these  questions  relate  specifically  to  the  products  surveyed. 

The  following  questions  were  chosen  because  they  address  one  or  more  of 
the  critical  study  areas:  preferred  medium  and  format  standards,  permanent 
public  accessibility,  permanent  retention,  user  fees,  commercial  licensing  of 
search  and  retrieval  software,  and  authenticity. 


Preferred  Medium  and  Study  Question  1:  What  combinations  of  preferred  medium  standards  are 
Format  Standards  currently  used  by  the  respondents? 

The  agencies  surveyed  are  creating  and  using  (in  descending  order): 

•  Products  both  in  paper  format  and  on  the  web. 

•  Products  both  in  CD-ROM  and  the  web. 

•  Products  both  in  paper  and  CD-ROM. 

Since  most  of  the  agencies  surveyed  create  products  in  more  than  one 
medium,  what  combinations  of  preferred  mediums  are  they  using? 

Of  the  respondents  who  indicated  that  paper  was  a  medium  used  and  the 
respondents  who  reported  that  CD-ROM  was  a  medium  used,  only 
19  percent  reported  that  they  are  using  both  paper  and  CD-ROM  products 
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(table  28).  Table  29  shows  that  of  the  respondents  who  reported  that  they 
use  CD-ROM,  and  those  who  reported  that  they  use  the  web  as  a  medium, 
21  percent  use  both  CD-ROM  and  the  web  as  mediums.  However,  of  the 
respondents  who  reported  using  paper  and  the  respondents  who  reported 
using  the  web  as  a  medium,  64  percent  use  both  paper  and  the  web  (table 
30).  Therefore,  the  respondents  surveyed  are  creating  and  using  products 
both  in  paper  format  and  on  the  web  much  more  often  than  they  are  creating 
and  using  products  in  CD-ROM  and  the  web.  An  even  smaller  percentage 
of  products  is  being  created  in  paper  and  in  CD-ROM.  This  confirms  the 
earlier  finding  that  paper  and  the  web  are  the  preferred  mediums  used  by  the 
agencies  surveyed,  but  provides  additional  information  about  the 
combinations  of  mediums  used. 


Table  28. 


Number  and  percent  crosstabulations  of  products 
in  both  paper  and  CD-ROM  formats  


Paper 

CD-ROM 

Total 

Yes 

No 

Number 

Percent 

Number 

Percent 

Number 

Percent 

Yes  

45 

18.6 

132 

54.5 

177 

73.1 

No  

25 

10.3 

40 

16.5 

65 

26.8 

Total  

70 

28.9 

172 

71.0 

242 

99.9 

NOTE:  Percents  may  not  add  to  100  because  of  rounding. 

SOURCE:  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product 
Assessment  Questionnaire,  1998. 


Table  29. 

Number  and  percent  crosstabulations  of  products 


in  both  CD-RO 

VI  and  web  formats 

CD-ROM 

Web 

Total 

Yes 

No 

Number  Percent 

Number  Percent 

Number  Percent 

Yes   51  21.1  19  7.9  70  29.0 

No   153  63.2  19  7.9         172  71.1 

Total   204  84.3  38  15.8        242  100.1 


NOTE:  Percents  may  not  add  to  100  because  of  rounding. 

SOURCE:  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product 
Assessment  Questionnaire,  1998. 
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Table  30. 


Number  and  percent  crosstabulations  of  products 
in  both  paper  and  web  formats  


Paper 

Web 

Total 

Yes 

No 

Number 

Percent 

Number 

Percent 

Number 

Percent 

Yes  

155 

64.0 

22 

9.1 

177 

73.1 

No  

V49 

20.2 

16 

6.6 

65 

26.8 

Total  

204 

84.2 

38 

15.7 

242 

99.9 

NOTE:  Percents  may  not  add  to  100  because  of  rounding. 

SOURCE:  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product 
Assessment  Questionnaire,  1998. 


Study  Question  2:  What  combinations  of  preferred  format  standards  are 
used  by  the  respondents? 

The  respondents  are  slightly  more  likely  to  use  HTML  in  combination 
with  PDF  than  they  are  to  use  HTML  together  with  GIF.  However, 
they  are  almost  as  likely  to  use  HTML,  GIF,  and  ASCH  together  as 
they  are  to  use  HTML,  PDF,  and  ASCn  together. 

Of  the  respondents  who  reported  using  HTML  as  a  tagged  markup  format, 
and  those  who  reporting  using  PDF  as  an  image  format,  39  percent  reported 
the  use  of  both  HTML  and  PDF  (table  31).  Of  the  respondents  who 
checked  HTML,  and  those  who  checked  GIF  as  an  image  format, 
36  percent  checked  that  they  used  HTML  in  combination  with  GIF  (table 
32),  slightly  less  than  those  who  used  HTML  and  PDF  in  combination. 
Since  PDF  is  the  preferred  image  format  used  by  agencies  (table  4),  this  is 
not  an  unexpected  finding. 


Table  31. 

Number  and  percent  crosstabulations  of  products 


in  both  HTML  and  PDF  formats 


HTML 

PDF 

Total 

Yes 

No 

Number 

Percent 

Number 

Percent 

Number 

Percent 

Yes  

94 

38.8 

63 

26.0 

157 

64.8 

No  

38 

15.7 

46 

19.0 

84 

34.7 

Total  

132 

54.5 

109 

45.0 

241 

99.5 

NOTE:  Percents  may  not  add  to  100  because  of  rounding. 

SOURCE:  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product 
Assessment  Questionnaire,  1998. 
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Table  32. 

Number  and  percent  crosstabulations  of  products 


in  both  HTML  and  GIF  formats 


HTML 

GIF 

Total 

Yes 

No 

Number 

Percent 

Number 

Percent 

Number 

Percent 

Yes  

86 

35.7 

70 

29.0 

156 

64.7 

No  

13 

5.4 

71 

29.5 

84 

34.9 

Total  

99 

41.1 

141 

58.5 

240 

99.6 

NOTE:  Percents  may  not  add  to  100  because  of  rounding. 

SOURCE:  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product 
Assessment  Questionnaire,  1998. 


However,  when  the  formats  are  used  in  combinations  of  three,  it  appears 
that  respondents  are  almost  as  likely  to  use  HTML,  GIF,  and  ASCII 
(21  percent)  together  as  they  are  to  use  HTML,  PDF,  and  ASCII 
(22  percent)  together  (tables  33  and  34). 


Table  33. 


Number  and  percent  of  products  that  use  HTML 
with  GIF  and  ASCII  formats 


Format 

Number 

Percent 

Both  GIF  and  ASCII  

50 

20.7 

GIF  only  

36 

14.9 

ASCII  only  

27 

11.2 

SOURCE:  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product 
Assessment  Questionnaire,  1998. 


Table  34. 


Number  and  percent  of  products  that  use  HTML 
with  PDF  and  ASCH  formats 


Format 

Number 

Percent 

Both  PDF  and  ASCII  

53 

21.9 

PDF  only  

41 

16.9 

ASCII  only  

25 

10.3 

SOURCE:  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product 
Assessment  Questionnaire,  1998. 
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Public  Access  to  Products      Study  Question  3:  If  a  product  is  permanently  accessible,  is  it  also  likely 

to  be  scheduled  for  retention  with  the  National  Archives  and 
Records  Administration  (NARA)? 

No,  the  majority  of  products  surveyed  that  are  permanently  accessible 
are  not  likely  to  also  be  scheduled  for  permanent  retention  with  NARA. 

Permanent  public  accessibility  and  permanent  record  retention  are  two 
distinct  concepts.  GPO,  through  the  FDLP,  has  a  historical  commitment  to 
permanent  accessibility  of  paper  products,  and  now  to  electronic  products. 
To  that  end,  GPO  requests  that  agencies  provide  information  products  in  all 
mediums  to  GPO  and  work  with  GPO  and  Federal  depository  libraries  to 
provide  permanent  public  accessibility  to  electronic  products.  Agencies  are 
responsible  for  transferring  those  products  that  are  scheduled  as  permanent 
records  to  NARA.  However,  not  all  records  that  are  scheduled  for 
permanent  retention  by  NARA  are  products  within  the  scope  of  the  FDLP. 
For  such  records,  permanent  public  accessibility  through  the  FDLP  is  not 
an  issue. 

Of  the  respondents  who  said  yes,  the  product  is  permanently  accessible,  and 
the  respondents  who  reported  their  product  is  scheduled  for  retention  with 
NARA,  only  25  percent  reported  that  the  product  is  both  permanently 
accessible  and  also  scheduled  for  retention  with  NARA  (table  35).  The 
majority  of  products  that  are  publicly  accessible  are  not  likely  to  also  be 
scheduled  for  retention  with  NARA.  While  there  is  not  information  from 
the  survey  data  to  identify  reasons  for  this  situation,  some  possibilities  are 
that: 

•  the  product  is  not  a  permanent  or  official  record  of  the  U.S. 
Government  as  defined  by  Federal  Records  legislation. 

•  the  product  is  in  a  format  that  is  accepted  by  GPO  but  in  a  format  that 
NARA  does  not  currently  accept,  and  therefore  could  not  be  transferred 
to  NARA. 

•  agencies  are  overlooking  this  important  part  of  the  information  life 
cycle  of  electronic  products. 


Table  35. 


Number  and  percent  crosstabulations  of  products  that  are  permanently  public  accessible  and 
scheduled  for  retention  with  the  National  Archives  and  Records  Administration  (NARA) 


Permanent  public  access 

Product  scheduled  with  NARA 

Total 

Yes 

No 

Don't  know 

Number 

Percent 

Number 

Percent 

Number 

Percent 

Number 

Percent 

Yes  

61 

25.2 

113 

46.7 

3 

1.2 

177 

73.1 

No  

21 

8.7 

41 

16.9 

3 

1.2 

65 

26.8 

Total  

82 

33.9 

154 

63.6 

6 

2.4 

242 

99.9 

NOTE:  Percents  may  not  add  to  100  because  of  rounding. 

SOURCE:  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product  Assessment  Questionnaire,  1998. 
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Study  Question  4:  Is  the  licensing  of  search  and  retrieval  software  likely 
to  be  a  barrier  to  unrestricted  public  access? 

No,  for  the  products  surveyed,  the  licensing  of  commercial  search  and 
retrieval  software  by  the  agency  does  not  appear  to  be  a  barrier  to 
unrestricted  (no  fee)  use. 

Of  the  respondents  who  reported  that  they  license  commercial  search  and 
retrieval  software  for  their  products,  and  those  who  reported  that  all  users 
are  charged  a  fee  for  the  products,  only  2  percent  who  license  commercial 
search  and  retrieval  software  also  charge  a  fee  for  all  users  (table  36).  A 
slightly  larger  number  of  respondents  (4  percent)  who  use  commercial 
search  retrieval  software  for  their  products  also  charge  a  fee  for  some 
users.  Twenty-five  percent  of  respondents  who  license  search  and  retrieval 
software  for  their  products  charge  no  user  fees. 


Table  36. 


Number  and  percent  crosstabulations  for  products  with  licensed  commercial  search  and  retrieval 
software  and  user  fees  charged  for  the  product  


Licensed  commercial  search  and 
retrieval  software 

User  fees  charged  for  product 

Total 

Yes,  for  all  users 

No,  for  some  users 

No 

Number 

Percent 

Number 

Percent 

Number 

Percent 

Number 

Percent 

Yes  

5 

2.1 

10 

4.1 

61 

25.2 

76 

31.4 

No  

16 

6.6 

38 

15.7 

112 

46.3 

166 

68.6 

Total  

21 

8.7 

48 

19.8 

173 

71.5 

242 

100.0 

NOTE:  Percents  may  not  add  to  100  because  of  rounding. 

SOURCE:  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product  Assessment  Questionnaire,  1998. 
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Study  Question  5:  Are  respondents  who  have  purchased  commercial 
search  and  retrieval  software  for  their  products  also  transferring 
the  products  to  NARA  ? 

No,  based  on  the  products  surveyed  here,  respondents  are  not 
transferring  permanent  records  to  NARA  for  products  in  which  they 
have  purchased  commercial  search  and  retrieval  software. 

Of  the  respondents  who  reported  issuing  commercial  search  and  retrieval 
software,  and  those  who  reported  scheduling  products  for  permanent 
retention  with  NARA,  only  about  10  percent  who  have  purchased 
commercial  software  for  products  have  also  scheduled  their  products  for 
permanent  retention  with  NARA  (table  37). 


Table  37. 


Number  and  percent  crosstabulations  of  those  products  with  licensed  commercial  search 
and  retrieval  software  and  the  product  is  scheduled  for  permanent  retention  by  the 
National  Archives  and  Records  Administration  (NARA)  


Product  scheduled  for  permanent  retention  by  NARA 

Licensed  commercial  search 

Total 

Yes 

No 

Don  t  know 

and  retrieval  software 

Number 

Percent 

Number 

Percent 

Number 

Percent 

Number 

Percent 

Yes  

24 

9.9 

52 

21.5 

0 

0.0 

76 

31.4 

No  

58 

24.0 

102 

42.1 

6 

2.5 

166 

68.6 

Total  

82 

33.9 

154 

63.3 

6 

2.5 

242 

100.0 

NOTE:  Percents  may  not  add  to  100  because  of  rounding. 

SOURCE:  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product  Assessment  Questionnaire,  1998. 
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Other  Issues:  Authenticity  Study  Question  6:  If  an  agency  ensures  authenticity,  is  it  also  likely  to 
and  Metadata  provide  permanent  public  access  to  the  product  or  do  agencies 

rely  on  another  agency  to  provide  permanent  public  access? 

Yes,  based  on  the  products  surveyed,  agency  respondents  who  ensure 
authenticity  for  their  products  are  also  more  likely  to  provide 
permanent  access  to  them  directly,  rather  than  through  another  agency. 

Of  those  respondents  who  reported  they  ensure  authenticity  and  those  who 
reported  they  provide  direct  permanent  public  access  to  their  products, 
47  percent  both  ensure  authenticity  for  their  products  and  provide  direct 
permanent  access  to  them  (table  38).  However,  only  close  to  14  percent  of 
the  respondents  who  reported  they  ensure  authenticity  for  their  products 
also  reported  that  another  agency  provides  permanent  public  access  to  the 
product  (table  39). 


Table  38. 

Number  and  percent  crosstabulations  of  those  products  for  which  agencies 
ensure  authenticity  and  permanent  public  access 

Agency  ensures  authenticity 

Permanent  public  access 
provided  by  the  agency 

Total 

Yes 

No 

Number 

Percent 

Number 

Percent 

Number 

Percent 

Yes  

113 

46.7 

41 

16.9 

154 

63.6 

No  

63 

26.0 

23 

9.5 

86 

35.5 

Don't  know  

1 

0.4 

1 

0.4 

2 

0.8 

Total  

177 

73.1 

65 

26.8 

242 

99.9 

NOTE:  Percents  may  not  add  to  100  because  of  rounding. 

SOURCE:  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product  Assessment  Questionnaire,  1998. 


Table  39. 

Number  and  percent  crosstabulations  of  those  products  for  which 

agencies  ensure  authenticity  and  another  agency  provides  permanent  public  access 

Another  agency  provides 

Agency  ensures  authenticity 

permanent  public  access 

Total 

Yes 

No 

Number 

Percent 

Number 

Percent 

Number 

Percent 

Yes  

33 

13.6 

121 

50.0 

154 

63.6 

No  

17 

7.0 

69 

28.5 

86 

35.5 

Don't  know  

1 

0.4 

1 

0.4 

2 

0.8 

Total  

51 

21.0 

191 

78.9 

242 

99.9 

NOTE.  Percents  may  not  add  to  100  because  of  rounding. 

SOURCE:  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product  Assessment  Questionnaire,  1998. 
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Study  Question  7:  Are  online  products  hosted  by  the  agency  that  created 
it  more  likely  to  have  a  metadata  record  than  products  hosted  by 
another  agency? 

Yes,  based  on  the  products  surveyed,  those  that  are  hosted  by  the 
agency  that  created  it  are  more  likely  to  have  a  metadata  record  than 
those  hosted  by  another  agency. 

Tables  40  and  41  show  that  almost  20  percent  of  the  products  that  are 
hosted  by  an  agency  also  have  a  metadata  record,  while  only  7  percent  of 
the  products  that  are  hosted  by  another  agency  also  have  a  metadata  record. 


Table  40. 


Number  and  percent  crosstabulations  of  products  that  are  hosted  by  the  agency  and  have  a  metadata 
record 


Products  hosted  by  the  agency 

Metadata  record 

Yes 

No 

Don't  know 

Total 

Number 

Percent 

Number 

Percent 

Number 

Percent 

Number 

Percent 

Yes  

48 

19.8 

141 

58.3 

10 

4.1 

199 

82.2 

No  

17 

7.0 

24 

9.9 

1 

0.4 

42 

17.3 

Total  

65 

26.8 

165 

68.2 

11 

4.5 

241 

99.5 

NOTE:  Percents  may  not  add  to  100  because  of  rounding. 

SOURCE.  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product  Assessment  Questionnaire,  1998. 


Table  41. 


Number  and  percent  crosstabulations  of  products  that  are  hosted 
by  another  agency  and  have  a  metadata  record  


Products  hosted  by  another 
agency 

Metadata  record 

Yes 

No 

Don't  know 

Total 

Number 

Percent 

Number 

Percent 

Number 

Percent 

Number 

Percent 

Yes  

17 

7.0 

25 

10.3 

0 

0.0 

42 

17.3 

No  

48 

19.8 

140 

57.9 

11 

4.5 

199 

82.2 

Total  

65 

26.8 

165 

68.2 

11 

4.5 

241 

99.5 

NOTE:  Percents  may  not  add  to  100  because  of  rounding. 

SOURCE:  National  Commission  on  Libraries  and  Information  Science,  Government  Information  Product  Assessment  Questionnaire,  1998. 
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4 


Qualitative 
Findings 


This  section  of  the  report  highlights  the  qualitative  findings  from  the  three 
site  visits  with  Federal  depository  libraries,  five  agency  meetings,  and  six 
expert  interviews.  Appendices  F  through  H  include  interview  questions  and 
detailed  responses  from  the  site  visits  to  depository  libraries  (F),  agency 
meetings  (G),  and  expert  interviews  (H). 


Site  Visits  to  Federal 
Depository  Libraries 


The  purpose  of  the  site  visits  to  the  three  depository  libraries  was  to  identify 
the  key  issues  and  concerns  librarians  have  about  providing  public  access  to 
electronic  Government  information  products  through  the  Federal  Depository 
Library  Program.  (See  Appendix  F  for  a  complete  list  of  questions  posed  to 
librarians.) 


The  site  visits  were  held  with  one  regional  depository  library  and  two 
selective  depository  libraries  in  the  Washington,  D.C.,  metropolitan  area.  It 
is  important  to  note  that  the  three  libraries  visited  may  not  be  representative 
of  all  depository  libraries  in  terms  of  the  geographical  location  and  library 
user  characteristics  (e.g.,  education  level,  socioeconomic  status,  etc.). 
Therefore,  readers  are  cautioned  about  generalizing  these  observations  to  all 
depository  libraries.  Highlights  of  the  three  librarians'  responses  are 
provided  below.  Appendix  F  contains  a  detailed  description  of  the 
librarians'  responses  to  the  interview  questions. 


Highlights  of  Site  Visits         User  Needs  and  Concerns 
to  Three  Depository 

Libraries  •    Librarians  interviewed  noted  that  the  general  public  is  still  more 

comfortable  using  Government  information  products  in  paper  and 
microfiche  than  they  are  using  the  Internet.  Patrons  (and  librarians)  are 
least  comfortable  using  products  on  CD-ROM. 

•  Librarians  expressed  concern  about  the  difficulty  patrons  experience  in 
accessing  Government-produced  CD-ROMs  that  are  not  standardized. 
They  reported  that  the  search  and  retrieval  software  is  different  for  each 
CD,  CD-ROMs  often  have  no  installation  instructions  or  user 
documentation,  and  they  are  not  user-friendly. 

•  Librarians  indicated  that  some  users  are  still  intimidated  by  electronic 
mediums  and  computers.  Most  users  ask  librarians  to  help  them  search 
for  materials  on  the  web  and  frequently  need  help  downloading  large 
files. 

•  Librarians  noted  that  since  most  Government  websites  only  contain  the 
most  recent  information,  they  are  concerned  about  users  having 
permanent  public  access  to  retrospective  Government  information  on 
the  web  in  the  future. 
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Librarians'  Concerns:  User  Fees,  Hardware,  Training,  and  Costs 

•  Although  none  of  the  libraries  visited  currently  charge  fees  for  printing 
materials  from  the  Internet  or  CD-ROMs,  all  three  librarians  are  either 
considering  charging  fees  or  are  planning  to  charge  fees  and  expressed 
concerns  about  how  this  will  affect  their  patrons. 

•  Users  do  not  have  access  to  enough  workstations,  so  the  libraries  must 
limit  use.  Also,  if  libraries  had  additional  money  for  hardware,  they 
would  order  hardware  in  support  of  CD-ROMs  (e.g.,  a  new  CD-ROM 
server  and  an  18-disk  CD  changer).  (Even  though  CD-ROM  is  the 
least  preferred  medium  and  declining  in  number  in  the  FDLP.) 

•  All  librarians  interviewed  expressed  concerns  about  finding  time  and 
money  to  train  librarians  and  staff,  especially  on  using  CD-ROM 
products,  but  also  on  downloading  files,  effectively  searching  the 
Internet  for  Government  information,  and  creating  and  maintaining  web 
pages.  They  welcome  any  additional  training  on  using  GPO  Access, 
Geographic  Information  Systems,  etc. 

•  Time  and  money  permitting,  librarians  expressed  interest  in  establishing 
partnerships  with  GPO  and  other  Government  agencies  to  put  some 
retrospective  online  Government  information  on  their  servers  so  users 
can  have  reliable  access  to  it  in  the  future.  In  addition,  librarians  would 
like  to  provide  outreach  to  public  schools,  community  centers,  etc.,  to 
educate  students  and  adults  about  the  wide  variety  of  valuable 
information  available  from  the  Federal  Government. 

•  One  librarian  expressed  strong  feelings  about  the  need  for  Congress  to 
provide  long-term  financial  support  to  Federal  depository  libraries  so 
they  can  continue  to  provide  permanent  public  access  to  digital 
materials.  This  librarian's  perspective  was  that  the  cost  to  provide 
access  to  electronic  Government  information  is  steadily  increasing. 


AeeilCV  Meetings  Meetings  were  held  with  four  agencies  between  September  14  through 

J  September  24,  1998: 

•  Department  of  Health  and  Human  Services 

•  Environmental  Protection  Agency 

•  U.S.  Department  of  Education 

•  U.S.  Department  of  Commerce 

Although  meetings  also  were  held  with  the  U.S.  Supreme  Court  and  the 
National  Archives  and  Records  Administration  (NARA),  these  two  agencies 
did  not  respond  to  the  agency  discussion  questions  in  the  agency  meetings; 
they   chose   to   discuss   the   survey   questionnaire   only.  However, 
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Lewis  Bellardo,  the  Deputy  Archivist  of  the  United  States,  sent  in  written 
responses  to  the  discussion  questions  (Appendix  G). 

The  National  Commission  on  Libraries  and  Information  Science  (NCLIS) 
provided  discussion  questions  for  the  agency  meetings;  Westat  modified 
some  of  the  questions  with  NCLIS's  approval.  The  purpose  of  the  agency 
meetings  was  to  supplement  survey  data  by  collecting  more  general 
information  on  electronic  Government  information  products  that  are  not 
product-specific.  For  example,  one  of  the  survey  objectives  is  to  assess  the 
cost-effectiveness  and  usefulness  of  preferred  medium  and  format 
standards,  an  issue  that  was  not  directly  addressed  on  the  survey.  In 
addition,  the  agency  meetings  afforded  NCLIS,  GPO,  and  Westat  an 
opportunity  to  review  the  survey  questionnaire  with  agency  respondents  and 
to  address  any  questions  they  might  have. 

Highlights  of  the  agency  meetings  are  provided  below.  For  a  more  detailed 
summary  of  the  responses  to  the  12  questions  posed  to  agencies,  see 
Appendix  G. 

Agency  Meeting  Highlights    Preferred  Mediums  and  Formats 

•  Agencies  interviewed  reported  using  the  same  preferred  medium  and 
format  standards  as  those  reported  by  survey  respondents:  web,  CD- 
ROM,  bulletin  board;  HTML,  PDF,  and  ASCII.  Additional  preferred 
formats  mentioned  by  agency  representatives  include  TIFF,  JPEG,  and 
Lotus/Domino. 

•  All  agencies  are  exploring  a  wide  range  of  innovative  and  creative  web 
design  approaches  including  the  use  of  SQL,  Oracle,  ColdFusion,  and 
animated  GIFs.  Some  examples  of  ways  in  which  agencies  are  utilizing 
web  technologies  include  data  warehousing,  interactive  GIS, 
multimedia  CD-ROM,  live  "real-time"  web  casting  of  selected 
speeches,  and  real-time  forecasting  of  air  pollution  levels  for  22  states. 

•  Four  of  the  five  agencies  have  guidelines  or  "best  practices"  for 
presentation  and  organization  of  products  or  publications  on  the  web. 
Most  of  the  guidelines  discuss  preferred  formats  for  some  types  of 
products.  The  most  common  problem  experienced  by  the  agencies  in 
this  regard  is  compliance  issues  (i.e.,  encouraging  personnel  to  adhere 
to  them). 

•  There  are  some  trends  for  migrating  certain  families  of  products  to  the 
web  for  newsletters,  training  manuals,  annual  reports,  and  conference 
proceedings  and  presentations. 

•  Agencies  consider  many  factors  when  making  decisions  to  create/retain 
products  in  more  than  one  medium:  budget,  cost,  accessibility  to  users, 
and     size     of     audience     the     product     reaches.  The 
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decision-making  process  varies  from  agency  to  agency  and  sub-unit  to 
sub-unit. 


Assessing  User  Needs 

•  All  agencies  reported  involving  users  in  testing  and  evaluating  the 
usefulness  of  the  web  and  CD-ROM  products.  The  most  frequently 
used  assessment  methods  are  focus  groups,  videotaping  of  users,  and 
online  user  surveys.  Agencies  are  using  the  results  of  these  evaluation 
methods  to  add  and  change  some  formats  and  mediums  as  well  as 
content. 

•  Four  of  the  five  agencies  interviewed  reported  that  they  maintain  some 
type  of  GILS  records  to  help  the  public  locate  their  information 
resources. 

Information  Life  Cycle  Management,  Permanent  Public  Access,  and 
Permanent  Retention 

•  No  agencies  are  addressing  the  following  key  information  resources 
management  issues:  permanent  public  access,  information  life  cycle 
management,  and  permanent  retention.  (The  expert  interviews  provide 
some  insight  into  the  reasons  that  agencies  are  not  addressing  these 
issues.  See  the  summary  section  of  this  report.) 

Cost-Effectiveness  of  Various  Mediums  and  Formats 

•  No  agencies  have  conducted  a  formal  cost-benefit  analysis  for  creating 
products  in  formats  and  mediums  for  distribution  to  the  Federal 
Depository  Library  Program.  Generally,  agency  representatives 
reported  it  costs  less  to  create  products  for  the  web  because  they  can 
avoid  production,  printing,  and  distribution  costs  for  paper  and  CD- 
ROM  products. 


The  interviews  with  six  experts  also  enriched  and  supplemented  the  survey 
findings.  Since  the  interviews  were  conducted  after  the  site  visits  and 
agency  meetings,  they  were  helpful  in  providing  a  broad  context  within 
which  the  survey  findings  could  be  viewed. 

The  expert  interviews  were  conducted  between  October  27  and  November 
24,  1998.  Telephone  interviews  were  held  with  two  webmasters,  two 
preservation  specialists,  and  two  professors  of  information  resources 
management.  These  experts  were  selected  from  a  list  provided  by  the 
NCLIS.  Highlights  from  each  set  of  interviews  are  provided  here.  (See 
Appendix  H  for  a  detailed  summary  of  each  telephone  interview.) 


Expert  Interviews 
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Interviews  With  Highlights  from  interview  with  webmasters  Jerry  Malitz,  National  Center 

Webmasters  for  Education  Statistics  (NCES),  and  Linda  Wallace,  the  Internal  Revenue 

Service  (IRS),  on  October  27,  1998. 

Preferred  Formats 

•  The  IRS,  unlike  the  other  agencies  surveyed,  primarily  uses  SGML, 
followed  by  PDF,  HTML,  and  Postscript.  They  train  their  authors  to 
use  SGML  because  they  consider  it  "intelligent  data"  that  can 
automatically  generate  other  formats  (e.g.,  web,  BBS,  Fax  on  Demand) 
through  templates  and  filters.  All  NCES  publications  are  in  PDF,  then 
HTML  (optional);  they  rarely  put  an  entire  publication  in  HTML 
format  only. 

•  The  IRS  has  conducted  a  cost-benefit  analysis  of  the  costs  of  delivering 
requests  through  different  formats.  They  have  found  that  it  costs  $3  per 
phone  call  to  fill  a  request,  1  cent  to  access  their  Internet  site  for  forms, 
etc.,  and  $2.50  to  make  a  CD-ROM  containing  5  years  of  IRS 
publications. 

•  IRS  indicates  that  it  provides  permanent  public  access  to  tax 
information  online  for  5  years,  and  from  their  "core  repository  library" 
for  about  14  years,  but  not  for  every  application.  However,  this  5  to  14 
years  means  that  IRS  provides  current  but  not  permanent  public  access 
to  their  Government  electronic  products. 

•  All  IRS  documents  are  ADA-compliant,  online  searchable,  and 
downloadable. 

User  Needs 

•  Both  IRS  and  NCES  assess  and  evaluate  the  effectiveness  of  their  web 
sites  with  advisory  groups  (IRS),  or  for  NCES,  through  an  Internet 
Working  Group  made  up  of  representatives  from  each  program  area. 

•  Both  agencies  have  GILS  records. 


Highlights  from  interview  with  preservation  specialists  Evelyn  Frangakis 
from  the  National  Agricultural  Library  (NAL)  and  Abby  Smith  from  the 
Council  on  Libraries  and  Information  Resources  (CLIR),  November  10, 
1998. 

Goals  of  Preservation 

•  It  is  useful  to  think  about  preservation  goals  such  as  enhancing  the 
long-term  preservation  of  and  access  to  information  of  enduring  value 
for  as  long  into  the  future  as  possible. 

•  There  is  no  standard  accepted  method  of  ensuring  long-term  access  to 
digital  information.   It  may  be  more  accurate  to  say  that  one  of  the 


Interviews  With 
Preservation  Specialists 
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primary  goals  of  preservation  is  to  set  up  systems  that  "sustain 
predictable  levels  of  loss." 

Barriers  to  Preservation  of  Digital  Materials 

•  The  concept  of  preservation  in  the  traditional  preservation  world 
examines  the  concept  of  permanence,  but  in  the  print  world  the  concept 
of  permanence  relates  to  chemical  inertness  and  mechanical  durability. 
These  concepts  do  not  translate  easily  into  a  digital  world. 

•  There  are  two  problems  with  digital  preservation:  (1)  media  in  which 
information  resides  may  be  unstable;  and  (2)  software/hardware 
configurations  on  which  information  is  stored  becomes  obsolete  so 
quickly  that  even  when  one  migrates  information  from  one  system  to 
another,  much  of  the  data  and  functionality  are  lost. 

•  Other  barriers  to  digital  preservation  include  that  it  is  difficult  to 
understand  what  we  can  and  cannot  do  under  current  copyright  law, 
and  any  transmission  link  is  as  strong  as  the  weakest  link.  The  weak 
link  in  the  transmission  of  electronic  information  is  human  beings,  not 
technology  (e.g.,  no  one  agency  or  organization  has  stepped  forward  to 
address  issues  like  information  life  cycle  management).  Preservation  of 
information  must  be  thought  about  at  the  creation  stage,  not  after  the 
information  has  been  collected  and  disseminated. 

•  One  of  the  core  infrastructure  problems  is  the  need  to  create  a  failsafe 
archives  mechanism  for  materials  that  disappear  from  the  web. 

Current  Preservation  Models  and  Initiatives 

•  NAL  and  partner  institutions  are  implementing  a  model  for  permanent 
public  access  and  preservation  of  agricultural  literature  that  addresses 
all  the  key  issues  in  information  resources  management:  inventory  and 
life  cycle  of  information,  permanent  public  access,  technical 
requirements,  and  user  access  and  retrieval.  (NAL  is  one  of  the  few 
examples  for  ensuring  a  failsafe  archives  for  preservation  of 
agricultural  literature.) 

CLIR  Initiatives 

•  CLIR  commissioned  a  report  by  Jeff  Rothenberg  from  RAND 
Corporation  on  emulation.  (Emulation  is  the  process  of  imitating  one 
system  with  another  so  both  accept  the  same  data,  execute  the  same 
programs,  and  achieve  the  same  results.) 

•  CLIR  commissioned  an  analysis  of  migrating  file  formats  to  do  a  risk 
assessment  associated  with  those  file  formats  during  migration. 
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•  CLIR  identified  a  computer  scientist  at  Carnegie  Mellon  University, 
John  Ockerbloom,  who  has  developed  a  system  of  file  conversion  called 
TOM  (Typed  Object  Model),  a  type  of  migration  that  converts  web- 
based  materials  to  different  file  formats. 


Interviews  With  Highlights  from  interviews  with  John  Bertot,  November  18,  and  Charles 

Information  Resources  McClure,  November  28,  1998.  (These  two  telephone  interviews  were  held 
Management  Specialists  separately.) 

Barriers  to  Successful  Implementation  of  Information  Resources 
Management  Initiatives 

•  Agencies  are  struggling  with  issues  such  as  permanent  public  access, 
information  life  cycle  management,  and  permanent  retention  due  to  a 
general  lack  of  information  resources  management  (IRM),  as  well  as 
organizational  policy  integration  for  Federal  Government  legislation 
and  initiatives. 

•  Agencies  do  not  view  information  as  a  strategic  resource  that  is  directly 
related  to  agency  missions.  Most  Government  IRM  initiatives  focus  on 
the  technology  side  of  IRM  because  it  is  tangible. 

•  Sometimes  smaller  agencies  are  more  successful  in  implementing  IRM 
initiatives  due  to  fewer  organizational  and  communication  barriers  to 
working  collaboratively. 

•  The  Information  Technology  Management  and  Reform  Act  of  1 996  did 
little  to  clarify  the  role  of  the  CIO  and  IRM  staff,  so  agencies  are  now 
struggling  with  what  to  do  with  these  functions. 

•  Agency  resources  are  now  almost  exclusively  devoted  to  Y2K  efforts 
with  little  time  and  resources  left  to  devote  to  IRM,  standards,  and 
operability. 

•  Staffing  and  training  are  critical  for  both  IRM  and  CIO  staff. 

•  Challenges  for  agencies  in  the  next  few  years  include  how  to  coordinate 
information  technology  and  information  technology  management, 
interoperability  and  standards  that  cut  across  agencies,  and  education 
and  training  of  staff. 
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5 


Discussion  of 
Quantitative 
and 

Qualitative 
Findings 


This  section  synthesizes,  integrates,  and  discusses  issues  and  the  major 
themes  that  emerge  from  the  survey  and  the  qualitative  data  collection 
activities,  including  the  interviews  with  Federal  depository  librarians, 
agency  personnel,  and  other  experts,  and  the  literature  review.  The  section 
is  arranged  by  the  following  key  study  issues: 


Preferred  mediums  and  formats, 
Evaluating  websites, 

Cost-effectiveness  of  formats  and  mediums, 
Depository  library  needs, 
Public  access  (public  domain  and  user  fees), 
Permanent  public  access  and  preservation. 


Preferred  Mediums 
and  Format 
Standards 


Survey  respondents  and  agency  representatives  reported  they  most  often  use 
the  following  mediums: 

•  Paper 

•  Web 

•  CD-ROM 

•  Bulletin  board  systems  (to  a  lesser  degree) 

Both  respondents  and  representatives  also  reported  use  of  the  following 
formats: 


HTML 

PDF 

GIF 

ASCII 

TIFF 


However,  most  agencies  whose  products  were  surveyed  use  these  mediums 
and  formats  as  a  common  agency  practice,  rather  than  as  an  agency 
mandate.  In  addition,  agency  representatives  and  webmasters  reported  they 
use  SGML,  Oracle  (with  ColdFusion  or  SQL),  JPEG,  and  TIFF  because 
these  formats  meet  the  information  needs  of  their  individual  constituents  or 
are  used  in  some  of  their  creative  web  approaches.  The  IRS  is  one  of  the 
few  agencies  interviewed  that  uses  SGML.  IRS'  Linda  Wallace,  one  of  the 
webmasters  who  served  as  an  expert  consultant  for  this  project,  indicates 
that  most  agencies  do  not  use  SGML  because  it  is  difficult  to  use.  But 
Wallace  noted  that  IRS  uses  SGML  because  it  is 
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much  more  robust,  and  it  is  easy  to  change  a  document  format  to  match 
customer  needs  (e.g.,  tax  law  information  for  consumers  and  for  lawyers). 
(See  Appendix  H  for  detailed  notes  on  the  telephone  interview  with  Linda 
Wallace.) 

A  few  survey  respondents  indicated  they  are  planning  to  change  to  or  add 
XML  or  other  object-oriented  formats.  XML  may  be  appealing  to  some 
agencies  because  data  can  be  stored  in  a  format  provided  by  XML  that  is 
transferable  to  a  v  wide  range  of  hardware  and  software  environments 
(Bryan,  1998,  p.  14).  In  addition,  according  to  Stuart  Culshaw,  XML 
makes  it  easier  for  authors  to  produce  documents  for  many  different  output 
mediums  (i.e.,  paper,  online  help,  web)  from  a  single  source  (Culshaw, 
1998,  p.  7). 

Most  of  the  agency  representatives  who  participated  in  the  meetings  also 
reported  that  their  agencies  have  established  written  guidelines  or  "best 
practices"  that  specify  preferred  formats  for  the  presentation  of  information 
on  the  web.  Even  though  these  guidelines  are  not  agency- mandated,  they 
seem  to  be  a  common  agency  practice.  Several  of  the  agencies  interviewed 
indicated  they  have  modified  or  adopted  their  agency  guidelines  from  the 
guidelines  established  by  the  Federal  Web  Consortium  in  1996. 

The  Consortium,  founded  in  1 994  by  the  National  Science  Foundation  and 
the  U.S.  Nuclear  Regulatory  Commission,  established  guidelines  with  other 
Government  agencies  (see  http://www.dtic.mil/staff/cthomps/  guidelines/). 
The  guidelines  provide  suggestions  to  help  the  Federal  community 
accomplish  agency  missions  to  improve  services  to  customers.  Consortium 
guidelines  cover  a  wide  range  of  topics  including: 

•  Home  page  checklist  (content,  navigation/organization,  style/markup); 

•  File  formats  (i.e.,  agencies  should  not  be  restricted  to  proprietary 
formats  such  as  WordPerfect,  Microsoft  Word,  SAS,  PDF); 

•  Rationale  for  using  certain  kinds  of  formats  such  as  HTML,  GIF,  and 
JPEG; 

•  Guidelines  for  formats  to  be  used  for  downloading  or  display  (e.g., 
HTML,  GIF,  JPEG,  PDF,  Postscript);  and 

•  Emerging  standards. 


Agency  representatives  indicated  that  one  of  their  biggest  challenges  is  to 
convince  personnel  from  all  program  areas  to  follow  the  agency's  internal 
guidelines  when  creating  products  for  the  web.  Another  challenge  for 
agencies  is  to  consolidate  web  guidelines  from  different  agency  sub-units  so 
they  are  complementary  rather  than  contradictory. 
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Agency  representatives,  per  OMB  Circular  A- 130  and  the  Government 
Performance  and  Results  Act  of  1993,  are  assessing  the  usefulness  of  their 
websites  and  CD-ROMs  as  part  of  a  larger  effort  to  measure  program 
effectiveness.  Focus  groups,  online  customer  surveys,  and  videotaping  of 
customers  online  are  the  most  common  ways  in  which  agencies  evaluate  and 
test  products  on  their  websites. 

One  objective  of  the  evaluation  is  to  test  both  formats  and  web  approaches. 
Based  on  the  evaluation  results,  agencies  may  change  or  add  formats.  For 
example,  one  agency,  after  testing  their  site  with  children,  eliminated  PDF 
files  on  the  site  and  made  it  more  interactive.  Another  agency  made  the 
decision  to  keep  their  BBS  because  many  of  their  international  users  do  not 
have  ready  access  to  the  web.  One  agency  webmaster  indicated  that  the 
needs  of  their  business  clients,  who  participate  on  their  advisory  board,  help 
drive  their  format  needs.  A  fourth  agency  stores  its  documents  in  TIFF 
format  for  image  and  textual  data.  As  customers  request  documents,  the 
agency  converts  them  to  PDF  so  customers  can  download  the  material.  A 
fifth  agency  created  a  simple  set  of  rules  for  producing  CD-ROMs  based 
upon  user  input:  keep  it  simple  to  use,  intuitive,  and  self-tutorial. 

User  needs  for  easy  access  to  electronic  information  products  will  continue 
to  affect  how  agencies  make  decisions  about  formats  and  mediums.  Bertot 
and  McClure  suggest  that  more  agencies  should  continue  to  monitor  the 
information  needs  of  the  public  as  well  as  targeted  constituencies  to  enhance 
current  access  to  electronic  Government  information  products  (Bertot  and 
McClure,  1997,  p.  288). 


Evaluating 
Websites 


Cost-Effectiveness 
of  Formats 
and  Mediums 


None  of  the  agency  representatives  who  attended  the  agency  meetings  has 
conducted  a  formal  cost-benefit  analyses  for  producing  or  creating  products 
in  preferred  or  emerging  formats,  mediums,  or  online  approaches  for 
distribution  to  the  FDLP.  Most  agencies  reported  that  migrating  products 
to  the  web  substantially  reduces  printing  and  distribution  costs  associated 
with  paper  mediums.  However,  the  crosstabs  in  tables  28-29  reveal  that 
many  Government  products  are  still  produced  in  more  than  one  medium  and 
often  in  more  than  one  format.  Providing  permanent  public  access  to 
electronic  mediums  ultimately  may  exceed  the  one-time  costs  associated 
with  producing  and  distributing  the  same  information  in  print  or  microform 
(GPO,  1996,  p.  24  and  A71-A74). 


In  her  role  as  Chief,  Electronic  Information  Services,  at  the  Internal 
Revenue  Service,  Linda  Wallace  has  analyzed  the  costs  of  delivering 
documents  to  customers  (Appendix  H).  She  found  that: 


It  costs  IRS  $3  per  call  for  the  public  to  call  into  their  toll-free  number 
and  for  IRS  to  fill  the  request. 
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•  The  cost  to  IRS  for  the  public  to  use  the  Internet  to  access  and  use  the 
forms  is  1  cent,  a  difference  of  300  to  1.  (However,  this  shifts  the  cost 
to  the  public,  who  must  have  access  to  the  Internet.) 

•  It  costs  IRS  $2.50  to  make  and  distribute  to  all  public  libraries 
(including  the  Federal  depository  libraries)  each  CD-ROM  containing  5 
years  of  tax  forms,  instructions,  and  publications. 

Based  on  these  numbers,  the  IRS  has  made  some  internal  decisions  about 
where  they  will  focus  their  resources  and  time  in  order  to  reach  the 
maximum  number  of  customers  in  the  most  cost-effective  manner. 

Since  depository  librarians  serve  as  the  intermediary  between  the  users  and 
electronic  information  products,  their  observations  and  experiences  about 
user  and  library  needs  are  critical.  In  general,  the  five  agencies  interviewed 
focused  on  public  users  or  their  target  audiences  rather  than  depository 
library  users  when  discussing  usage  of  their  electronic  Government 
information  products. 

First,  the  librarians  interviewed  emphasized  that  many  patrons  still  prefer 
Government  information  in  paper  mediums,  followed  by  the  web  and  then 
CD-ROM.  The  respondents  surveyed  indicated  that  many  of  their  products 
are  produced  both  in  paper  and  on  the  web. 

Second,  librarians  expressed  concerns  about  lack  of  standardization  for 
producing  Government  CD-ROMs.  One  agency  representative  indicated 
that  they  are  undertaking  several  initiatives  to  make  their  CD-ROMs  more 
user-friendly  by  making  them  as  intuitive  as  possible  and  incorporating  a 
user  testing  component  into  the  production  schedule. 

A  third  important  concern  for  the  librarians  interviewed  is  the  rising  cost  of 
computer  hardware  and  the  simultaneous  rise  in  user  expectations  for  state- 
of-the-art  computer  workstations.  Although  the  three  libraries  recently 
received  updated  computer  workstations  that  met  or  exceeded  the 
recommended  minimum  guidelines  for  depository  libraries,  they  are 
beginning  to  change  their  policies  on  access  to  workstations  by  placing  a 
time  limit  on  their  use. 

A  fourth  issue  concerns  the  rising  costs  to  purchase  and  maintain  new 
equipment,  which  have  caused  depository  librarians  to  reconsider  their 
policies  on  charging  printing  fees.  One  librarian  indicated  that  their  library 
already  charges  patrons  for  photocopying  materials;  this  change  is  not 
dramatic,  but  it  does  affect  the  concept  of  no-fee  access  when  an 
overwhelming  number  of  products  are  offered  on  the  Internet. 

Fifth,  time  and  resources  to  train  library  staff  (and  patrons)  on  how  to  use 
the  new  technology  (i.e.,  how  to  download  files),  conduct  Internet  searches, 
design  and  develop  their  own  websites,  and  load,  search,  and  use 


Depository  Library 
Needs 
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CD-ROMs  are  major  concerns  expressed  by  the  depository  librarians 
interviewed.  The  fact  that  Government  information  exists  in  a  variety  of 
mediums  and  formats  only  increases  rather  than  diminishes  the  need  for 
training. 

Finally,  all  librarians  are  troubled  by  how  GPO,  the  FDLP,  and 
Government  agencies  will  address  the  problems  of  permanent  public  access 
to  electronic  information  products  that  are  constantly  being  replaced  and 
updated  by  new  ones.  In  addition,  the  preservation  of  retrospective 
electronic  Government  information  is  an  issue  of  concern. 


Public  Access  The  survey  data  revealed  that  15  percent  of  the  products  surveyed  are  not  in 

the  public  domain,  for  all  or  part  of  the  product.  In  addition,  user  fees  are 
charged  for  30  percent  of  the  products.  These  data  suggest  that  these  two 
critical  public  access  goals  have  not  yet  been  achieved. 


Permanent  Public  Access 
to  and  Permanent 
Retention  of  Electronic 
Government  Information 


Perhaps  more  than  any  other  issues,  permanent  public  access  and 
preservation  pose  two  of  the  greatest  challenges  to  the  FDLP,  and 
ultimately  to  the  public.  Each  of  the  experts  raised  different  issues  and 
shared  various  perspectives  about  these  issues.  It  might  be  helpful  here  to 
summarize  their  perspectives  and  describe  initiatives  underway  to  address 
the  problems  associated  with  the  provision  of  permanent  public  access  and 
preservation. 


Most  of  the  survey  respondents  indicated  that  permanent  access  is  currently 
provided  for  the  products  surveyed,  although  most  of  the  responses 
indicated  that  this  concept  is  not  fully  understood  and  that  access  is  not 
provided  by  the  agency  responsible  for  the  product.  Instead,  they  are 
relying  on  GPO,  Federal  depository  libraries,  the  National  Technical 
Information  Service,  or  other  agencies  to  provide  this  permanent  public 
access.  In  its  policy  and  planning  document,  Managing  the  FDLP 
Electronic  Collection  (see  http://www.access.gpo.gov/ 

su_docs/dpos/ecp lan. html),  GPO  states  that  "the  'first-level'  collection 
management  activity  depends  upon  knowledge  that  the  products  exist.  In 
order  to  ensure  current  and  permanent  access,  GPO  will  ...rely  on 
notification  from  and  outreach  to  other  agencies  and  notification  from  the 
depository  library  community." 

The  responses  of  agency  representatives  on  the  issue  of  permanent  public 
access  may  provide  additional  information  about  the  problem.  Most 
agency  representatives  said  their  agencies  had  not  discussed  the  issue  or 
were  exploring  the  issue  to  see  how  it  should  be  addressed,  and  they 
indicated  that  they  did  not  understand  the  concept  of  permanent  public 
access  in  relation  to  permanent  retention.  The  one  exception  was  the 
representative  from  National  Archives  and  Records  Administration,  who  is 
clear  about  the  agency's  role  to  provide  permanent  public  access  to  its  own 
products. 
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It  might  be  helpful  here  to  clarify  the  distinctions  between  the  two  concepts. 
GPO's  definition  of  permanent  public  access  "means  that  electronic 
Government  information  products  within  the  scope  of  the  FDLP  remain 
available  for  continuous,  no-fee  public  access  through  the  program"  (GPO, 
1998,  p.  19). 

Lewis  Ballardo,  deputy  archivist  of  the  United  States,  in  a  recent  article  in 
the  Washington  Post  (March  12,  1999,  p.  A01)  stated  that  the  problem  of 
digital  preservation  must  be  addressed  "or  memory  will  be  lost  for  the  latter 
half  of  the  20th  century."  In  addition,  Bellardo,  in  a  written  response  to 
agency  questions,  articulated  agency  responsibilities  to  GPO  for  permanent 
public  access  and  to  NARA  for  permanent  retention.  GPO  will  accept 
products  in  all  mediums  to  provide  continuous,  no-fee  public  access,  if 
notified  by  agencies  that  access  is  being  discontinued.  Agencies  are 
responsible  for  transferring  those  products  that  are  scheduled  as  permanent 
records  (official  records  as  defined  by  Federal  Records  legislation)  to 
NARA. 

Linda  Wallace  described  the  IRS'  methods  for  providing  current  public 
access  to  their  materials.  Using  SGML  format,  the  IRS  has  built  and 
maintains  a  core  knowledge  repository  to  generate  media  output  in  any 
application  to  respond  to  customer  needs.  The  repository  maintains 
materials  for  14  years,  but  not  for  every  application.  In  addition,  all  tax 
forms,  publications,  instructional  materials,  etc.,  are  available  online  for  5 
years.  Since  none  of  the  agencies  interviewed  is  providing  permanent 
public  access  to  its  products,  it  was  useful  to  ask  two  information  resources 
management  experts,  John  Bertot  and  Charles  McClure,  to  provide  some 
larger  context  within  which  the  problem  can  be  viewed. 


Perspectives  on  Permanent 
Public  Access  and 
Information  Life  Cycle 
Management  from 
Information  Resources 
Management  Experts 


Both  Bertot  and  McClure  have  extensively  studied  and  taught  information 
resources  management  (IRM).  They  attribute  the  lack  of  successful 
implementation  of  IRM  initiatives  in  the  Federal  Government  to  the 
following  factors: 

•  There  is  no  comprehensive  integrated  Federal  IRM  policy;  current 
policies  do  not  adequately  address  permanent  public  access, 
information  life  cycle,  and  electronic  records  management. 

•  There  is  no  strategic  vision  of  IRM  by  agencies;  information  is  not 
viewed  as  a  resource  that  should  be  used  to  accomplish  agency 
missions. 

•  Most  agency  initiatives  focus  on  the  technology  side  of  IRM  because  it 
is  tangible. 

•  Most  agencies  are  targeting  their  information  technology  resources 
toward  Y2K  efforts. 

•  There  is  no  clear  distinction  between  the  role  of  information  resources 
managers  and  CIOs. 
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•    There  is  no  ongoing  training  for  IRM  and  CIO  staff. 

(See  Appendix  H  for  detailed  notes  on  telephone  interviews  with  Bertot  and 
McClure,  and  Bertot  and  McClure,  1997,  pp.  280-282.) 

There  are  many  IRM  policy  instruments  from  the  Paperwork  Reduction  Act 
of  1980  and  1986,  OMB  Circular  A-130  (1985;  and  1993  and  1994 
revisions)  through  the  Information  Technology  Management  Reform  Act 
(ITMRA)  of  1996  and  Executive  Order  13011  (July  1996).  But  Bertot  and 
McClure  (1998)  emphasize  that  there  is  still  a  lack  of  an  integrated  policy. 
For  example,  in  their  focus  group  with  IRM  managers,  Bertot  and  McClure 
noted  that  managers  felt  that  the  Paperwork  Reduction  Act  assumed  that  the 
managers  understood  and  knew  how  to  manage  the  information  life  cycle, 
but  they  agreed  that  agency  management  at  all  levels  never  grasped  the 
concept  either  in  theory  or  in  practice.  In  addition,  the  ITMRA  that  created 
a  position  for  an  agency-based  CIO  to  oversee  agency  IRM  activities  and  to 
provide  education  for  agency  IRM  personnel  and  agency  managers  (among 
other  things)  does  not  clarify  the  relationships  between  and  among  CIOs 
and  IRM  managers.  Consequently,  it  is  ambiguous  about  whether  the 
agency  CIO's  organization  replaces,  incorporates,  or  is  separate  from 
current  agency  IRM  functions. 

Given  this  larger  context,  it  is  not  surprising  that  IRM  issues  such  as 
information  life  cycle  management,  preservation,  and  permanent  public 
access  have  not  been  adequately  addressed.  Conventional  organizational 
barriers  such  as  size,  culture,  poor  communication  and  interaction  across 
and  within  agencies,  and  lack  of  ongoing,  strategic  training  for  IRM  and 
CIO  staff  may  exacerbate  these  challenges  faced  by  agencies  (telephone 
interview  with  Bertot,  Appendix  H).  (As  an  example,  McClure  states  that 
IRM  graduate  students'  degrees  are  useful  for  about  1-2  years  after  they 
graduate.  After  that,  their  skills  are  50  percent  out  of  date;  telephone 
interview  with  McClure,  Appendix  H.)  Several  experts  are  involved  in 
initiatives  that  address  some  of  these  important  IRM  issues. 


Current  Initiatives  on  Several  agencies,  organizations,  and  Federal  depository  libraries  with 

Permanent  Public  Access  partner  institutions  are  exploring  ways  to  address  the  problems  of 
and  Permanent  Retention      permanent  public  access,  preservation,  and  electronic  records  management. 

Appendix  H  contains  more  detailed  information  about  each  of  these 

initiatives  that  will  be  summarized  here. 

Abby  Smith  from  the  Council  on  Libraries  and  Information  Resources 
(CLIR),  and  Evelyn  Frangakis  from  the  National  Agricultural  Library 
(NAL)  are  supporting  research  and  testing  models  for  permanent  public 
access  and  preservation.  The  three  CLIR  initiatives  are  described  below: 
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•  A  commissioned  report  by  Jeff  Rothenberg  from  RAND  Corporation  on 
emulation.  The  report  has  been  completed  and  was  published 
in  January  1999.  The  report  describes  the  weaknesses  of  migration  and 
the  strengths  of  emulation  and  sets  up  a  research  agenda  to  develop 
emulation.  (Log  onto  publications  on  CLIR  site  for  a  summary  of 
Rothenberg' s  report:  http://www.clir.org/pubs/reports/ 
rothenberg/contents .  html . ) 

•  A  commissioned  analysis  of  migrating  file  formats  to  support  a  risk 
assessment  associated  with  those  file  formats  during  migration.  The 
study  by  Cornell  University,  using  data  from  the  Mann  (agricultural) 
Library,  will  use  numeric  file  formats  and  databases  and  text  formats. 
The  report,  to  be  finished  by  September  1 999,  will  include  analysis  and 
a  template  that  others  can  use  for  doing  a  risk  assessment  of  migration 
of  those  file  formats. 

•  CLIR  is  working  with  John  Ockerbloom,  a  computer  scientist  at 
Carnegie  Mellon  University  (CMU)  who  has  developed  a  system  of  file 
conversion  called  TOM  (Typed  Object  Model).  (See  www.es. 
emu .  edu/afs/cs .  emu .  edu/user/spok/www/ defense/ index,  html) .  CLIR 
would  like  to  see  if  they  can  bring  his  concepts  into  fuller  application. 

Smith  describes  NAL  and  its  efforts  to  provide  permanent  public  access  and 
to  preserve  agricultural  literature  as  one  of  few  examples  where  a  failsafe 
archives  might  work,  partly  because  NAL  is  a  national  library  dedicated  to 
one  type  of  literature.  Evelyn  Frangakis  is  involved  in  NAL's  efforts  to 
develop  its  own  preservation  program  that  includes  a  traditional 
preservation  program  and  digital  efforts.  Their  digital  efforts  are  two- 
pronged: 

•  Conversion  of  brittle  paper  materials  into  digital  products  by  working 
with  the  best  available  guidelines  to  implement  good  preservation 
practices.  They  will  make  this  digital  material  available  on  the  web. 

•  Development  of  a  program  to  preserve  USDA  digital  materials  (i.e., 
materials  that  are  born  digitally). 

In  addition,  Frangakis  is  also  involved  in  a  national  effort  to  preserve 
agricultural  literature.  The  U.S.  Department  of  Agriculture's  Digital 
Publications  Preservation  Steering  Committee  was  established  in  1998  to 
oversee  the  implementation  of  the  plan,  A  Framework  for  the  Preservation 
of  and  Permanent  Public  Access  to  USDA  Digital  Publications.  This 
group  met  for  the  first  time  in  October  1998.  The  plan  may  serve  as  a 
model  that  other  agencies  or  institutions  can  adapt.  USDA  is  incorporating 
the  following  needs  and  considerations  into  its  framework: 

•  Inventory  and  life  cycle  management, 

•  Technical  requirements,  and 
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•  User  access  and  retrieval. 

USDA  is  moving  ahead  to  implement  the  plan.  The  USDA  CIO  accepted 
the  report,  and  under  Frangakis'  guidance,  NAL  established  a  national 
steering  committee  made  up  of  representatives  from  USDA  and  from 
agribusiness,  the  research  library  community,  the  U.S.  Agricultural 
Information  Network  (USATN),  Federal  partners,  etc.  The  group  will  meet 
on  a  quarterly  basis  for  the  first  2  years.  They  will  establish  test  groups  to 
explore  the  technical  and  funding  issues.  They  are  hoping  to  secure  funding 
for  a  pilot  project  to  test  the  framework  on  an  agency  within  USDA  to  see 
how  manageable  it  will  be  for  full-scale  implementation  (see  Appendix  H 
for  a  detailed  description  of  the  Framework). 

Finally,  GPO  has  established  partnerships  with  several  depository  libraries 
and  Federal  agencies  to  provide  permanent  public  access  to  remotely 
accessible  electronic  Government  information  products.  Three  such 
partnerships  include: 

•  Partnership  with  the  University  of  Illinois  at  Chicago's  Richard  J. 
Daley  Library  and  the  U.S.  Department  of  State  (DOS)  to  provide 
permanent  access  to  remotely  accessible  electronic  DOS  information 
products. 

•  An  Online  Computer  Library  Center/GPO  pilot  project  with  the  U.S. 
Department  of  Education/National  Library  of  Education  (NLE) 
provided  free  public  access  through  the  FDLP  to  remotely  accessible 
electronic  Educational  Resources  Information  Center  (ERIC) 
documents. 

•  A  project  with  the  Department  of  Energy  (DOE)  Office  of  Scientific 
and  Technical  Information  (OSTI)  to  provide  public  and  depository 
library  access  to  DOE  technical  reports  in  image  format  via  the  web 
service  called  "DOE  Information  Bridge"  (Aldrich,  1998). 

Preservation  specialists  Smith  and  Frangakis  noted  that  technology  is  not 
the  biggest  barrier  to  permanent  access  and  preservation;  the  human 
infrastructure  is  not  in  place  yet  that  would  ensure  permanent  access  and 
preservation  (telephone  interview  with  Smith  and  Frangakis,  Appendix  H). 
The  plans  and  initiatives  described  here,  coupled  with  the  recommendations 
for  training,  policy  integration,  and  support  for  best  practices  to  implement 
policies  are  a  few  of  the  strategic  actions  that  appropriate  agencies, 
libraries,  and  institutions  should  undertake  to  ensure  that  future  generations 
will  have  unrestricted,  no-fee  access  to  Government  information  in  all 
formats. 


Next  Steps  As  a  fohowup  effort,  NCLIS  indicated  that  they  will  use  these  findings  as  a 

point  of  departure  and  analyze  them  in  greater  depth.  It  is  expected  that 
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this  followup  effort  will  result  in  broad  conclusions  and  recommendations  to 
the  President  and  Congress  about  how  the  problems  and  challenges  revealed 
in  this  study  can  be  constructively  addressed. 
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Assessment  of  Electronic  Government  Information  Products 
Agency  Study  Coordinator  Meetings 
July  23,  1998,  and  August  4,  1998 

The  Benton  Foundation 
1634  I  Street,  N.W. 

11th  Floor 
Washington,  D.C. 


Agenda 


9:30  -  9:45       Background  and  Welcome 

Robert  S.  Willard,  Executive  Director,  National  Commission  on  Libraries 
and  Information  Science  (NCLIS) 

Francis  J.  Buckley,  Jr.,  Superintendent  of  Documents,  Government  Printing  Office  (GPO) 


9:45  -  10:15     Objectives,  Timetable,  and  Required  Actions 
Forest  Woody  Horton,  NCLIS  Study  Manager 


10:15-10:30    Coffee  Break 

10:30  -  12:15    Specific  Data  Requirements  (question-by-question  review  of  survey  instrument  using 
GPO  Access  as  a  specific  product  example) 


Gil  Baldwin,  GPO 
T.C.  Evans,  GPO 
Ric  Davis,  GPO 
Denise  Glover,  Westat 
Elizabeth  Farris,  Westat 
Steve  Fischer,  Westat 


12:15  Wrap-up  and  C losing 

Forest  Woody  Horton 

12:30  Adjourn 
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Agency  Coordinators  and  Other  Key  Officials 
 Agency  Coordinator(s) 


Legislative  Branch 


Library  of  Congress 
United  States  Congress 


Nancy  Davenport 
Maggie  Smith 

Eric  Peterson 


Judicial  Branch 

Administrative  Office  of  the  U.S.  Courts  Gloria  Malkin 


Supreme  Court  of  the  United  States 


Shelley  L.  Dowling 
Wilma  M.  Grant 


Department  of  Agriculture 
Department  of  Commerce 

Department  of  Defense 

Department  of  Education 

Department  of  Energy 
Department  of  Health  and  Human 

Department  of  the  Interior 

Department  of  Justice 

Department  of  Labor 


Executive  Branch 

Betty  Behal 

Vera  Whisenton 
Cynthia  Banicki 

William  Beyer 
Rick  Silva 

Chiquitta  Thomas 
Linda  Tague 

Karen  Spence 

Services  Deborah  Burris 

Fred  Wood 

Claude  Christensen 
Bob  Mehnert 

Regina  Byrd 
Chris  Rudy 
Dennis  Feldt 

Maureen  Hill 
Deborah  Klein 
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Agency  Coordinator(s) 


Department  of  State 
Department  of  Transportation 
Department  of  the  Treasury 


Executive  Branch  (continued) 

Colleen  Hope 
Dan  Clemmer 

Barbara  Post 
Robert  Zarnetske 

Gladys  Myatt 
Mike  Conklin 


Environmental  Protection  Agency 
Executive  Office  of  the  President 
General  Services  Administration 
National  Aeronautics  and  Space  Administration 
National  Archives  and  Records  Administration 
Securities  and  Exchange  Commission 
Smithsonian  Institution 
Social  Security  Administration 


Richard  Huffine 
Peter  Weiss 
Odessa  Brown 
Roland  Ridgeway 
Debra  Leahy 
Bert  Lee 
Robert  Schelin 
Terry  Hynes 
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Participating  Agencies  and  Products  Surveyed 


Survey 
status 

Legislative  Branch 


Library  of  Congress 

American  Memory:  Historical  Collections  CM 

Braille  Books  on  Disk  CM 

Cataloger's  Desktop  CM 

CD  BLND  CM 

Classification  Plus  CM 

Country  Studies  CM 

French  Poster  Art  CM 

THOMAS  CM 

Thomas:  A  Century  of  Lawmaking  for  a  New  Nation  CM 

Thomas:  Bill  Summary  and  Status  CM 

Thomas:  Committee  Reports  CM 

Thomas:  Congressional  Record  CM 

Thomas:  Early  Congressional  Documents  CM 

Thomas:  Enactment  of  a  Law  CM 

Thomas:  Federalist  Papers  CM 

Thomas:  Floor  Activities  for  the  House  &  Senate  CM 

Thomas:  House  Committee  Schedules  OS 

Thomas:  House  Roll  Call  Votes  CM 

Thomas:  How  Our  Laws  are  Made  CM 

Thomas:  Major  Legislation  CM 

Thomas:  Senate  Roll  Call  Votes  CM 

Thomas:  Text  of  Bills  CM 

United  States  Congress 

Congressional  Bills  CM 

Congressional  Directory  CM 

Congressional  Record  (bound  permanent)  CM 

Congressional  Record  (daily)  CM 

Hearings,  Reports,  &  Prints:  House  Agriculture  Committee  * 

Hearings,  Reports,  &  Prints:  House  Appropriations  Committee  * 


Hearings,  Reports,  &  Prints:  House  Education  and  Workforce  Committee 
Hearings,  Reports,  &  Prints:  House  Government  Reform  and  Oversight  Committee 


Hearings,  Reports,  &  Prints:  House  Judiciary  Committee  * 

Hearings,  Reports,  &  Prints:  Joint  Committee  on  Taxation  CM 

Hearings,  Reports,  &  Prints:  Senate  Appropriations  Committee  CM 

Hearings,  Reports,  &  Prints:  Senate  Armed  Services  Committee  CM 

Hearings,  Reports,  &  Prints:  Senate  Commerce  Committee  * 


Hearings,  Reports,  &  Prints:  Senate  Foreign  Relations  Committee 


*Survey  nonrespondent. 
CF  =  Complete  by  fax 
CM  =  Complete  by  mail 
CP  =  Complete  by  phone 
OS  =  Out-of-scope 
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Survey 
status 


Legislative  Branch  (continued) 


United  States  Congress  (continued) 

Hearings,  Reports,  &  Prints:  Senate  Judiciary  Committee  CM 

Hearings,  Reports,  &  Prints:  Rules  and  Administration  Committee  * 

Serial  Set  * 

United  States  Code  V  CM 

Judicial  Branch 

Administrative  Office  of  the  U.S.  Courts 

Federal  Rules  of  Civil  Procedures  CM 

Judicial  Business  of  the  United  States  Courts  CM 

The  Official  Bankruptcy  Forms  Collection  CM 

The  Third  Branch  CM 

Understanding  the  Federal  Courts  CM 

Supreme  Court  of  the  United  States 

Bench  Opinions  of  the  Supreme  Court  CM 

Rules  of  the  Supreme  Court  CM 

Slip  Opinions  of  the  Supreme  Court  CM 

U.S.  Reports  CM 

Executive  Branch 

Department  of  Agriculture 

Agent  Orange  CM 

AgExporter  CM 

AGRICOLA  Database  * 

Agricultural  Prices,  Monthly  CM 

Agricultural  Statistics  CM 

Aquaculture  CM 

Continuing  Survey  of  Food  Intake  By  Individuals  CM 

Crops  County  Data  CM 

Fact  Sheets  CM 

FAS  Hot  Country  Pages  CM 

Fire  Effects  Information  System  CM 

Forest  Land  Distribution  Data  for  the  U.  S.  CM 

Leaflets  CM 

Market  News  CF 

NAFTA  Agricultural  Fact  Sheets  CM 

Ornamental  Horticultural  Multimedia  Project  CM 


*Survey  nonrespondent. 
CF  =  Complete  by  fax 
CM  =  Complete  by  mail 
CP  =  Complete  by  phone 
OS  =  Out-of-scope 
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Survey 
status 

Executive  Branch  (continued) 


Department  of  Agriculture  (continued) 

Outlook  for  U.S.  Agricultural  Exports  CM 

Statistical  Bulletins  CM 

The  Plant  Genome  Database  Collaboration  CM 

Wildland  Fire  Assessment  System  CM 

World  Agricultural  Supply  and  Demand  Estimates  * 

Department  of  Commerce 

AgroBase  Database  * 

Cen-Data  OS 

CenStats  * 

Census  of  Population  and  Housing  * 

Commerce  Business  Daily/CBDNet  CM 

County  and  City  Data  Book  CM 

Current  Population  Reports  * 

Economic  Census  Reports  * 

Federal  Research  in  Progress  Database  * 

Geophysics  of  North  America  * 

GOV. Research-Center  * 

Imports/Exports  CD  CM 

Local  Climatalogical  Data  (for  states)  CM 

National  Trade  Data  Bank  CM 

NTIS  Database  * 

Regional  Economic  Information  System  (REIS)  CM 

Solar  Radio  Bursts  * 

Statistical  Abstract  of  the  United  States  CM 

STAT-US  A/Internet/State  of  the  Nation  CM 

STAT-USA  Newsletter  CM 

Survey  of  Current  Business  CM 

TIGER/Line  CM 

U.S.  Industry  and  Trade  Outlook  CM 

USA  Counties  CM 

Zip  Code  Area  CD-ROM  CM 

Department  of  Defense 

Airman  Magazine  CM 

Airpower  Journal  CM 

All  Hands  * 

Defense  Logistics  Agency  Publishing  System  CM 

Joint  Electronic  Library  CM 


♦Survey  nonrespondent. 
CF  =  Complete  by  fax 
CM  =  Complete  by  mail 
CP  =  Complete  by  phone 
OS  =  Out-of-scope 
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Survey 
status 

Executive  Branch  (continued) 


Department  of  Defense  (continued) 

Marines  Magazine  CM 

Notice  to  Mariners  CM 

Soldiers  Magazine  CM 

Technical  Manuals  V  CM 

Department  of  Education 

Condition  of  Education  CM 

Digest  of  Education  Statistics  CM 

Disability  Statistics  Abstract  CM 

Disability  Statistics  Report  CM 

EDsearch,  Education  Statistics  on  Disk  CM 

ERIC  Database  CM 

Guide  to  Education  Programs  CM 

Helping  Your  Child  (series)  CM 

NAEP  (Nat'l  Assessment  of  Educational  Progress)  Mathematics  CM 

National  Education  Goals  Report  CM 

Projections  of  Education  Statistics  CM 

Resources  in  Education  CM 

Student  Guide  CM 

Think  College?  Me?  Now?  CM 

Department  of  Energy 

Country  Analysis  Briefs  (CABs)  * 
DOE  Directives  * 

DOE  Information  Bridge  CM 

DOE  R&D  Project  CM 

Electric  Power  Annual  CM 

Electric  Power  Monthly  CM 
Emissions  of  Greenhouse  Gases  in  the  U.S.  * 
Energy  Consumption  &  Carbon  Emissions  by  Region  * 

Energy  Files  CM 

Energy  InfoDisc  CM 

Energy  Science  and  Technology  Database  CM 

International  Energy  Outlook  CM 
Monthly  Energy  Outlook  * 
National  Education  Goals  Report  * 

Petroleum  Supply  Annual  CM 

State  Energy  Data  Reports  CM 

State  Energy  Data  System  CM 


♦Survey  nonrespondent. 
CF  =  Complete  by  fax 
CM  =  Complete  by  mail 
CP  =  Complete  by  phone 
OS  =  Out-of-scope 
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Survey 
status 

Executive  Branch  (continued) 


Department  of  Energy  (continued) 

Weekly  Petroleum  Status  Report  CM 

World  Energy  Consumption  * 

Department  of  Health  and  Human  Serv  ices 

Annual  Pesticide  Residue  Monitoring  Report  CM 

Biosafety  in  Microbiological  and  Biomedical  Laboratories  CM 

CDC  Wonder  CM 

Clinical  Preventive  Services  CM 

EHP  (Environmental  Health  Perspectives)  CM 

FDA  Almanac  CM 

FDA  Compliance  Policy  Guide  CM 

HCFA  Health  Watch  CM 

HCFA's  Laws,  Regulations,  Manuals  * 

Healthcare  Cost  &  Utilization  Project  (HCUP-3)  CM 

Medicare  and  You  CM 

Medicare  Compare  CM 

Morbitity  &  Mortality  Weekly  report  CM 

National  Health  Interview  Survey,  State  Data  Files  CM 

NIOSHTIC  Database  CM 

PubMed  CM 

Research  Activities  (Monthly  Newsletter)  CM 

RTECS  Database  CM 

SETS:  Statistical  Export  &  Tabulation  System  CM 

Vital  and  Health  Statistics  (Rainbow  Series)  CM 

Department  of  the  Interior 

Contaminant  Information  Mgt  &  Analysis  System  CM 

Endangered  Species  Bulletin  CM 

Geographic  Names  Information  System  CM 

Metal  Industry  Indicators  (MTJ)  CM 

Mineral  Industry  Surveys  (monthly  &  quarterly)  CM 

Minerals  Yearbook  CM 

National  Survey  of  Fishing,  Hunting  &  Wildlife  &               •  * 

National  Wildlife  Refuge  System  Profiles  Database  CM 

New  Publications  of  the  Geological  Survey  CM 

NPS  Statistical  Abstract  CM 

Preservation  Briefs  (numbered  series)  * 

The  National  Register  Information  System  (NRIS)  CM 

Water  Resources  Abstracts  CM 


♦Survey  nonrespondent. 
CF  =  Complete  by  fax 
CM  =  Complete  by  mail 
CP  =  Complete  by  phone 
OS  =  Out-of-scope 
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Survey 
status 

Executive  Branch  (continued) 


Department  of  Justice 

Correctional  Populations  of  the  United  States  CF 

FBI  Law  Enforcement  Bulletin  CM 

FOIA  Annual  Report  CM 

Justice  Information  Center  Publications  v  CM 

NCJRS  Abstracts  Database  CM 

Prison  and  Jail  Inmates  CF 

Sourcebook  of  Criminal  Justice  Statistics  CF 

Uniform  Crime  Reports:  Crime  in  the  United  States  CM 

Department  of  Labor 

General  Wage  Determinations  Issued  Under  Davis-Bacon  * 

Monthly  Labor  Review  CM 

Occupational  Outlook  Handbook  CM 

OSHA  Documents  and  Files  * 

Department  of  State 

Background  Notes  CM 

Diplomatic  List  * 

Dispatch  CM 

Key  Officers  of  Foreign  Service  Posts  * 

Maximum  Travel  Per  Diem  Allowance  for  Foreign  Area  * 

Medical  Information  for  Americans  Traveling  Abroad  CF 

Treaties  and  Other  International  Acts  Series  * 

Department  of  Transportation 

FAA  Statistical  Handbook  CM 

Highway  Statistics  CM 

National  Transit  Database  CM 

National  Transportation  Statistics  CM 

Nationwide  Personal  Transportation  Survey  CM 

Rail  Waybill  Data  CM 

Transportation  Expressions  CM 

Transportation  Statistics  Annual  Report  CM 

Worldwide  Transportation  Directory  CM 

Department  of  Treasury 

A  Visitors  Guide  OS 

Arson  Investigations  Guide,  ATF  P  2220.1  * 

Bonded  Warehouse-Manual  for  Proprietors,  Importers  &  Customs  Officers  CM 


*Survey  nonrespondent. 
CF  =  Complete  by  fax 
CM  =  Complete  by  mail 
CP  =  Complete  by  phone 
OS  =  Out-of-scope 
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Executive  Branch  (continued) 


Department  of  Treasury  (continued) 

Business  Taxpayers  Information  Publications,  Pub  1 194.B  CM 

Buying  Treasury  Securities,  PD  P  0009  * 

Counterfeiting  &  Forgery  * 

Customs  Valuation  Encyclopedia,  1980-1997  * 

Daily  Treasury  Statements  CM 

Federal  Tax  Forms  CM 

Firearms  State  Laws  &  Published  Ordinances,  ATF  P  5300.5  CM 

Importing  a  Car  (Pub  520)  CM 

Know  Your  Money  * 

Payment  of  Tax  by  EFT,  ATF  P  5000.10  CM 

Quarterly  Journal  (Online)  CF 

Tables  for  Redemption  Values  for  US  Series  E  Savings  Bonds  &  Savings  Notes  CM 

Tables  of  Redemption  Values  for  US  Series  EE  Savings  Bonds  CM 

Tax  Guide  for  Small  Businesses,  Pub  334  * 

The  History  of  Money  OS 

The  Money  Story  CF 

Treasury  Bulletin  * 

U.S.  Mint  Gift  Collection  CF 

Your  Federal  Income  Tax,  Pub  1 7  CM 

Your  Guide  to  Federal  Firearms  Regulations,  ATF  P  5300.4  * 

Environmental  Protection  Agency 

Center  for  Environmental  Info.  &  Statistics  CM 

Consumer  Handbook  for  Reducing  Solid  Waste  CM 

Envirofacts  Warehouse  CM 

EPA  Online  Library  System  (OLS)  CP 

Federal  Register  Environmental  Subset  CM 

Guide  to  Environmental  Issues  CM 

Integrated  Risk  Information  System  Database  (IRIS)  CP 

National  Air  Quality  and  Emission  Trends  Report  CM 

National  Environmental  Publications  Information  CM 

National  Water  Quality  Inventory:  Biennial  Report  to  Congress  (305b  report)  CM 

Recycle  City  CM 

Sector  Facility  Indexing  Project  CM 

Setting  the  Record  Straight:  Secondhand  Smoke  is  a  Preventable  Health  Risk  CM 

Superfund  Hazardous  Waste  Site  Query  (CERCLIS  Data)  CM 

Surf  Your  Watershed  CM 

Test  Methods  for  Evaluating  Solid  Waste:  Physical/Chemical  Methods  (SW-846)  CM 


♦Survey  nonrespondent. 
CF  =  Complete  by  fax 
CM  =  Complete  by  mail 
CP  =  Complete  by  phone 
OS  =  Out-of-scope 
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Executive  Branch  (continued) 

Executive  Office  of  the  President 

Art  in  the  White  House  —  A  Nation's  Pride 
Best  Kept  Secrets  in  Government 
Budget  of  the  United  States 
FBIS  Publications  Reports  v 
From  Red  Tape  to  Results:  Creating  a  Government  & 
Interactive  Citizens'  Handbook 
NAICS  Manual 
OMB  Circulars 

Putting  Customers  1st,  Serving  the  American  Public 
Reinvention  Express 
The  White  House  for  Kids 
Virtual  Library 
White  House  Briefing  Room 
White  House  History  (web) 
White  House  Tour  (CD-ROM) 
World  Factbook 
World  News  Connection 


General  Services  Administration 

Catalog  of  Federal  Domestic  Assistance  CM 

Consumer  Information  Catalog  CF 

Consumer  Information  Series  OS 

Consumer  Resource  Handbook  CM 

Federal  Acquisition  Regulation  * 

Government  Registration  Service  CM 

MarkeTips  * 

U.S.  Gold  CM 

U.S.  Government  TDD/TTY  Directory  CM 

United  States  Online  Directories  * 

US  Government  Blue  Pages  Online  Directory  CM 

US  Real  Property  Sales  List  (online  title  varies)  CM 

National  Aeronautics  and  Space  Administration 

Aerospace  Medicine  and  Biology  CM 

CASI  Technical  Report  Server  CM 

NASA  Thesaurus  CM 

NASA  Video  Catalog  CM 

Patent  Abstracts  CM 

Scientific,  Technical  Aerospace  Reports  CM 


* 

CM 
* 

CM 
* 

* 

* 

CM 

CM 
* 

* 

* 

* 

* 

CM 
* 


*Survey  nonrespondent. 
CF  =  Complete  by  fax 
CM  =  Complete  by  mail 
CP  =  Complete  by  phone 
OS  =  Out-of-scope 
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Survey 
status 

Executive  Branch  (continued) 
National  Archives  and  Records  Administration 

Code  of  Federal  Regulations  CM 

Digital  Classroom  CM 

Emerging  Nation  CM 

Federal  Register  CM 

National  Archives  Information  Locator  (NAIL)  CM 

Online  Exhibit  Hall  CM 

Privacy  Act  Issuances  CM 

Public  Laws  (slip  laws)  CM 

United  States  Government  Manual  CM 

Weekly  Compilation  of  Presidential  Documents  CM 

Survey 
status 

Securities  and  Exchange  Commission 

Administrative  Proceedings  CM 
Cold  Calling  * 
Commission  Legal  Briefs  * 
Completed  Initiatives  of  Interest  to  Small  Business  * 
Directory  of  Companies  Filing  Annual  Reports  * 
EDGAR  Database  of  Corporate  Information  * 
Financial  Facts  Tool  Kit  * 
Investment  Fraud  &  Abuse  Travel  to  Cyberspace  * 

Litigation  Actions  and  Proceedings  Bulletin  OS 

Litigation  Releases  CM 
Official  List  of  Section  13  (f)  Securities  * 
Official  Summary  of  Security  Transactions  &  Holdings  * 
Pending  Initiatives  of  Interest  to  Small  Business  * 
Plain  English  Handbook:  How  to  Create  Clear  SEC  * 
SEC  Annual  Report  * 

SEC  Concept  and  Interpretative  Releases  CM 

SEC  Decisions  CM 

SEC  Docket  OS 

SEC  Final  Rules  CM 

SEC  News  Digest  (daily)  CM 

SEC  Opinions  CM 

SEC  Proposed  Rules  CM 

SEC  Special  Studies  CM 

Staff  Accounting  Bulletins  CM 

Trading  Suspensions  CM 


♦Survey  nonrespondent. 
CF  =  Complete  by  fax 
CM  =  Complete  by  mail 
CP  =  Complete  by  phone 
OS  =  Out-of-scope 
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Survey 
status 

Executive  Branch  (continued) 


Smithsonian  Institution 

Annals  of  the  Smithsonian  Institution 

CM 

Anthology  of  American  Folk  Music 

CM 

Gallery  Exhibition  Catalogues 

* 

SIRIS:  SI  Research  Information  System 

CM 

Smithsonian  (magazine)  V 

* 

Smithsonian  Contributions  to  Anthropology 

CM 

Smithsonian  Contributions  to  Botany 

CM 

Smithsonian  Contributions  to  Earth  Sciences 

CM 

Smithsonian  Contributions  to  Marine  Sciences 

CM 

^mitliQruiian  CrMitrihntinnc  tn  Pa lpnhiolocrv 
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CM 

Smithsonian  Contributions  to  Zoology 

CM 

Smithsonian  Folklife  Festival:  Culture  of,  by  &  .  .  . 

* 

Smithsonian  Folklife  Studies  Series 

CM 

Smithsonian  Institute  Research  Reports 

* 

Smithsonian  Photographs  Online 

* 

Smithsonian  Studies  in  Air  and  Space 

CM 

Smithsonian  Year 

* 

Social  Security  Administration 

Documents  Published  by  the  SSA  Historian 

CM 

Publications  Information  Pamphlets  and  Fax  Sheets 

CM 

Request  a  Personal  Earnings  &  Benefits  System 

CM 

SSA  Publications  on  CD-ROM 

CM 

*Survey  nonrespondent. 
CF  =  Complete  by  fax 
CM  =  Complete  by  mail 
CP  =  Complete  by  phone 
OS  =  Out-of-scope 
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United  Slates 
National  Commission  on 
Libraries  and  Information  science 


October  1998 


Dear  Agency  Coordinator: 

On  behalf  of  the  National  Commission  on  Libraries  and  Information  Science  (NCLIS)  and  the  Government 
Printing  Office,  we  would  like  to  thank  you  for  agreeing  to  distribute  this  survey  of  Government  Electronic 
Information  Products  to  agency  respondents. 

The  purpose  of  this  assessment  is  to:  (1)  identify  medium  and  format  standards  that  are  the  most 
appropriate  for  permanent  public  access,  (2)  assess  the  cost-effectiveness  and  usefulness  of  alternative 
medium  and  format  standards,  and  (3)  identify  public  and  private  medium  and  format  standards  that  are  or 
could  be  used  for  products  throughout  their  entire  information  life  cycle.  The  broader  objective  of  the 
survey  is  to  study  the  long-term  impacts  of  shifting  government  information  products  from  paper  and 
microform  mediums  to  federal  agency  web  sites  and  other  electronic  mediums.  We  want  to  ensure  that  as 
more  government  information  is  available  in  a  variety  of  electronic  mediums  and  formats,  the  American 
public  continues  to  have  free  and  easy  local  access  to  this  information  through  the  Federal  Depository 
Library  Program. 

We  are  requesting  that  you,  as  the  agency  coordinator,  promptly  distribute  the  questionnaires  to  the 
appropriate  personnel  in  your  agency.  If  you  have  any  questions  about  this  survey,  please  call  Denise 
Glover  at  Westat:  301-251-2269  or  toll-free  at  800-937-8281,  ext.  2269.  We  will  send  you  a  copy  of  the 
final  report  once  it  is  completed. 

The  following  suggested  procedures  and  instructions  will  assist  you  to  successfully  distribute  the  product 
questionnaires.  However,  some  agency  coordinators  have  decided  to  use  different  data  collection 
procedures.  If  you  are  one  of  those  coordinators,  it  is  essential  that  you  explain  your  procedures  to 
your  respondents. 

1 .  All  agency  coordinators  will  receive  a  full  packet  from  Westat  on  September  29  or  shortly  thereafter 
that  contains  a  cover  letter  and  an  information  copy  of  the  questionnaire.  The  coordinator's  packet  will 
also  include  a  sealed  packet  for  each  product.  Please  make  sure  you  have  a  packet  for  each  product  on 
your  final  list  of  product  selections.  If  you  are  missing  information  or  have  incorrect  information, 
please  call  Debbie  Alexander  at  Westat:  301-294-2088,  or  toll-free  at  800-937-8281,  ext.  2088. 

2.  The  product  packet  includes  a  cover  letter  to  the  respondent,  a  product  questionnaire,  a  glossary  of 
terms,  and  a  postage-paid  return  envelope. 

3.  Each  product  questionnaire  contains  the  following  preprinted  information:  a  five-digit  ID,  the  agency's 
name  and  submit,  and  the  name  of  the  product. 

4.  Upon  receipt  of  the  packet,  please  distribute  the  product  packets  to  the  appropriate  product  respondents 
in  your  agency. 

1  1  IO  Vermont  Avenue,  N.W.  Suite  820 
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5.  Due  to  the  specialized  nature  of  many  of  the  questions  asked  on  the  survey,  it  may  be  necessary  for 
product  respondents  to  consult  with  other  agency  personnel  such  as  records  managers,  information 
technology  staff,  planning  offices,  and  others  to  complete  the  survey  in  its  entirety.  We  recommend 
that  you  ask  product  respondents  to  leave  blank  any  questions  they  feel  uncomfortable  answering  and 
to  make  arrangements  for  someone  in  their  office  or  another  office  with  more  appropriate  knowledge 
and  expertise  to  answer  those  questions.  If  product  respondents  decide  to  take  this  action,  we  strongly 
suggest  that,  if  possible,  you  standardize  the  arrangements  in  advance  and  ensure  that  all  agency 
respondents  are  aware  of  your  procedures.  One  purpose  of  standardizing  arrangements  is  to  ensure 
that  product  respondents  do  not  lose  control  of  a  questionnaire  because  multiple  individuals  and 
offices  are  handling  it 

6.  Product  respondents  are  responsible  for  ensuring  that  all  questions  are  answered  on  their  respective 
questionnaires,  even  if  they  must  consult  with  other  personnel. 

7.  To  ensure  consistency  and  completeness,  you  might  want  to  ask  your  assigned  product  respondents  to 
allow  you  to  review  the  surveys  before  they  send  them  to  Westat.  You  might  also  request  that 
respondents  send  you  a  copy  of  the  completed  questionnaire  for  your  files. 

8.  Product  respondents  should  return  completed  questionnaires  directly  to  Westat  (not  to  you)  in  the 
enclosed  postage-paid  envelope.  If  product  questionnaires  are  not  completed  and  returned  to  Westat  by 
the  October  30,  1998  deadline,  Westat  will  attempt  to  follow  up  directly  with  the  designated  product 
respondent.  However,  if  respondent  information  is  unavailable  or  unknown,  Westat  will  contact  you 
for  followup. 

Respondents  are  requested  to  return  the  survey  by  October  30, 1998  by  mailing  it  in  the  postage-paid 
envelope  to: 

Denise  Glover 
TA  2064 
Westat,  Inc. 

1650  Research  Boulevard 
Rockville,  MD  20850 

Thank  you  very  much  for  your  assistance  and  cooperation.  We  appreciate  the  time  and  hard  work  you  have 
invested  in  the  coordination  activities. 

Sincerely, 

Robert  S.  Willard  Francis  J.  Buckley,  Jr. 

Executive  Director  Superintendent  of  Documents 

National  Commission  on  Libraries  and  Information  Science  Government  Printing  Office 
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United  states 
National  Com  mission  on 
Libraries  and  Information  Science 


January,  1999 

Dear  Respondent: 

On  behalf  of  the  National  Commission  on  Libraries  and  Information  Science  (NCLIS)  and  the  Government 
Printing  Office,  we  are  requesting  that  you  complete  this  survey  of  Government  Electronic  Information 
Products. 

The  purpose  of  this  assessment  is  to:  (1)  identify  medium  and  format  standards  that  are  the  most 
appropriate  for  permanent  public  access,  (2)  assess  the  cost-effectiveness  and  usefulness  of  alternative 
medium  and  format  standards,  and  (3)  identify  public  and  private  medium  and  format  standards  that  are  or 
could  be  used  for  products  throughout  their  entire  information  life  cycle.  The  broader  objective  of  the 
survey  is  to  study  the  long-term  impacts  of  shifting  government  information  products  from  paper  and 
microform  mediums  to  federal  agency  web  sites  and  other  electronic  mediums.  We  want  to  ensure  that  as 
more  government  information  is  available  in  a  variety  of  electronic  mediums  and  formats,  the  American 
public  continues  to  have  free  and  easy  local  access  to  this  information  through  the  Federal  Depository 
Library  Program. 

We  recognize  that  many  respondents  will  complete  surveys  for  more  than  one  agency  product.  We  also 
understand  the  burden  this  imposes  upon  you  and  appreciate  the  time  and  effort  you  will  devote  to 
completing  the  survey.  However,  we  want  to  underscore  the  importance  of  your  participation.  We  will 
send  you  a  copy  of  the  report  summarizing  the  data  from  these  surveys  once  it  is  completed. 

Please  carefully  read  all  of  the  following  suggested  procedures  and  instructions  that  will  assist  you  to 
successfully  complete  and  return  the  product  questionnaires.  However,  some  agency  coordinators  have 
decided  to  use  different  data  collection  procedures.  You  should  contact  your  coordinator  directly  to 
find  out  if  you  are  to  follow  the  procedures  outlined  here.    If  your  coordinator  has  made  other 
arrangements,  please  contact  him/her  to  determine  what  they  are. 

1.  Once  you  receive  your  packet(s)  from  your  agency  coordinator,  please  check  to  make  sure  each 
packet  contains  the  following  information  for  each  product  you  are  to  survey:  a  cover  letter, 
questionnaire,  glossary  of  terms,  and  postage-paid  return  envelope.  If  you  feel  you  are  missing 
information  or  have  incorrect  information,  please  first  contact  your  agency  coordinator.  (Your 
coordinator's  contact  information  is  available  on  the  NCLIS  web  site  at  www.nclis.gov.)  If 
your  coordinator  is  unable  to  provide  this  information,  call  Debbie  Alexander  at  Westat:  301- 
294-1088,  or  toll-free  at  800-937-8281,  ext.  2088. 

2.  Each  product  questionnaire  contains  the  following  preprinted  information:  a  five-digit  ID,  the 
agency's  name  and  subunit,  and  the  name  of  the  product. 
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3.  Due  to  the  specialized  nature  of  many  of  the  questions  asked  on  the  survey,  it  may  be  necessary 
for  you  to  consult  with  other  agency  personnel  such  as  records  managers,  information 
technology  staff,  planning  officers,  and  others  to  complete  the  survey  in  its  entirety.  We 
suggest  you  leave  blank  the  responses  to  questions  you  feel  uncomfortable  answering  and  then 
arrange  for  someone  in  your  office  or  another  office  with  more  appropriate  knowledge  and 
expertise  to  answer  those  questions.  Your  agency  coordinator  may  have  standardized  those 
arrangements  in  advance  and  made  you  are  aware  of  their  procedures.  The  purpose  of 
standardizing  arrangements  is  to  ensure  that  you  do  not  lose  control  of  a  questionnaire 
because  multiple  individuals  and  offices  are  handling  it. 

4.  Please  note  that  you  are  responsible  for  ensuring  that  all  questions  are  answered  on  your 
respective  questionnaires,  even  if  you  must  consult  with  other  personnel. 

5.  To  ensure  consistency  and  accuracy,  coordinators  might  request  that  they  be  allowed  to  review 
the  completed  surveys  before  you  return  them  to  Westat. 

6.  Return  completed  questionnaires  directly  to  Westat  in  the  enclosed  postage-paid  envelope  by 
January  15,  1999.  Your  coordinator  may  request  that  you  also  send  him/her  a  copy  of  the 
completed  questionnaire. 

7.  If  you  have  any  questions  about  this  survey,  please  call  Denise  Glover  at  Westat:  301-251- 
2269,  or  toll-free  at  800-937-8281,  ext.  2269. 

Please  return  the  questionnaire  by  January  15, 1999  by  mailing  it  to: 

Denise  Glover 

TA  2064 

Westat,  Inc. 

1650  Research  Blvd. 

Rockville,  MD  20850 

Thank  you  very  much  for  your  assistance  and  cooperation. 


Sincerely, 


Robert  S.  Willard 
Executive  Director 

National  Commission  on  Libraries  and  Information  Science 


Francis  J.  Buckley,  Jr. 
Superintendent  of  Documents 
Government  Printing  Office 
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NATIONAL  COMMISSION  ON  LIBRARIES  AND  INFORMATION  SCIENCE 


GOVERNMENT  INFORMATION 
PRODUCT  ASSESSMENT  QUESTIONNAIRE 


Directions:  This  form  is  to  be  completed  only  for  products  that  are  either  already  in  electronic  mediums  or  products  that  are 
to  be  migrated  to  electronic  mediums.  Do  not  complete  this  form  for  products  that  will  remain  in  paper  or  microform 
mediums  only.  A  product  is  defined  as  "a  Government  publication  or  other  work  of  the  United  States  Government  conveyed 
in  a  tangible  physical  medium  such  as  a  book,  CD-ROM,  etc.,  or  disseminated  through  an  electronic  Government  information 
service  and  intended  for  public  dissemination."  (See  enclosed  glossary  for  definitions  of  terms  used  throughout  this 
questionnaire.)  Complete  one  questionnaire  for  each  product.  (Please  note  that  a  Website  is  not  considered  a  product, 
although  products  might  be  on  a  Website.) 

A.       GENERAL  INFORMATION 

1.       Agency  Name:  

Sub-Unit: 


2.       Name  of  Product: 


3.       Brief  Description  of  Product: 


4.       URL  for  Product  Website: 


O  Check  if  no  Website 
B.       CURRENT  PRODUCT  PROFILE 

5.       How  is  this  product  used  by  the  end  user?  (Check  all  that  apply.) 

a.  Information  access  and  retrieval   0 1 

b.  Data  analysis  (e.g.,  to  support  analysis  by  end  user)    0 1 

c.  Other  (specify)   0 1 
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What  types  of  data  are  contained  within  this  product? 


In  Column  A,  indicate  which  type  of  data  this  product  contains.  (Check  all  that  apply.) 

In  Column  B,  indicate  the  primary  type  of  data  contained  in  this  product.  For  example,  if  you  checked  boxes  for 
items  b  and  f  in  column  A,  indicate  which  of  the  two  is  the  primary  type  of  data  by  placing  a  check  in  the  appropriate 
box  in  column  B. 


Type  of  data 

A.  Type  of  data 
contained 

B.  Primary 
type  of  data 
(Check  only 
one) 

(Check  all  that  apply) 

a.    Bibliographic  data 

□  ' 

b.    Graphical  data  (photos,  charts,  graphs,  tables,  drawings) 

□  2 

c.    Numerical  data 

□  3 

d.  Sound 

l  1  1 

□r 

I  1  A 

□  4 

e.    Spatial  data  (maps,  coordinate  files) 

□  5 

f.    Textual  data  (books,  serials,  reports) 

□  ' 

□  6 

g.  Video 

□' 

□  7 

h.    Multimedia  (sound,  video,  text,  graphics) 

□* 

□  8 

i.     Other  {specify) 

□' 

□  9 

E-4 


7a. 


In  what  mediums  is  this  product  publicly  available? 


In  Column  A,  indicate  which  type  of  medium  is  used.  (Check  all  that  apply.) 

In  Column  B,  for  each  medium  used,  indicate  whether  there  is  a  medium  standard  that  is  mandated  by  the  agency,  a 
common  practice  in  the  agency  (although  not  mandated),  other  (i.e.,  not  agency-mandated  standard  or  common 
agency  practice,  but  new  and  promising  and  beginning  to  be  used,  etc.),  OR  none  of  these. 
In  Column  C,  for  each  type  of  medium  used,  check  one  box  to  indicate  the  primary  type  used.  


Medium 


A.  Type  of 
mediums  used 


(Check  all  that 
apply) 


B.  Standard- 
Is  there... 


Agency- 
mandated 
standard 


Common 
agency 
practice 


Other 


None 


(Check  one  for  each  category) 


C.  Primary  type 
of  medium  used 
(Check  only  one) 


Pre-Electronic  Mediums 

a.  Paper  

b.  Microform  

c.  Other  (specify) 


Electronic  Mediums 
Magnetic  Mediums 

d.  Magnetic  tape  

e.  Floppy  diskette  

f.  Hard  drive  

g.  Other  {specify)  

Describe  medium  in  more  detail  

Optical  Mediums 

h.  CD-ROM  

i.  WORM  (Write  once,  read  many  disk) 

j.     DVD  (digital  video  disk)  , 

k.     Other  (specify)  

Describe  medium  in  more  detail 


Online  Mediums 

1.  Web  

m.  Gopher  

n.  Bulletin  Board  Systems, 
o.     Other  (specify)  


Describe  medium  in  more  detail 


Q1 
□  ' 


rij1 


o1 


□2 
□2 
□2 


□2 
□2 
□2 
□2 


□2 
□2 
□2 
□2 


□2 
□2 
□2 
□2 


□3 
□3 


□3 
□3 
□3 
□3 


□3 
□3 
□3 
□3 


□3 
□3 
□3 
□3 


□4 
□4 
□4 


□4 
□4 
□4 
□4 


□4 
□4 
□4 
□4 


□4 
□4 
□4 
□4 


7b.  If  you  checked  "Other"  category  in  Question  7a,  Column  B,  please  explain. 
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8a.      Which  of  the  following  types  of  formats  does  this  product  use? 

In  Column  A,  indicate  which  type  of  format  is  used.  (Check  all  that  apply.) 

In  Column  B,  for  each  format  used,  indicate  whether  there  is  a  format  standard  that  is  mandated  by  the  agency,  a 
common  practice  in  the  agency  although  not  mandated,  other  (i.e.,  not  agency-mandated  standard  or  common  agency 
practice,  but  new  and  promising  and  beginning  to  be  used,  etc.),  OR  none  of  these. 

In  Column  C,  for  each  type  of  format  used,  check  one  box  to  indicate  the  primary  type  that  is  used  within  each 
major  category  (e.g.,  database,  spreadsheet,  word  processing,  etc.). 


Format 

A.  Formats 
used? 

B.  Standard — 
Is  there... 

C.  Primary  type 
of  format  used 
(Check  one  in 
each  category) 

(Check  all  that 
ap^ply) 

Agency- 
mandated 
standard 

Common 
agency 
practice 

Other 

None 

(Check  one  for  each  category) 

Database 

a.  Oracle  

□  □□□□□ 

□  □□□□□ 

□  □□□□□ 

□  □□□□□ 

□  □□□□□ 

n2 

□3 

□4 

□5 
n6 

c.  dBase  

d.  WAIS  

e.  MARC  

f.     Other  (specify) 

Spreadsheet 

a.  Excel  

□2 

□  2 

□  2 

□3 
□3 
□  3 

□  4 

□  4 

□  4 

□' 
□2 
□3 

b.     Lotus  1-2-3  

c.     Other  (specify) 

Tagged  Markup 

a.  HTML  

□  □  □  □ 

□  □  □  □ 

□  2 

□  2 
□2 

□  2 

□  3 

□  3 

□  3 

□  3 

□  4 

□  4 

□  4 

□  4 

□  ' 
□2 
□3 
□4 

D.  AML  

c.  SGML  

d.     Other  (specify) 

Image 

a.  GIF  

□  □  □  □  □ 

□  □  □  □  □ 

□  2 

□  2 

□  2 

□  2 

□  2 

□3 

□  3 

□  3 

□  3 

□  3 

□  4 

□  4 

□  4 

□  4 

□  4 

□  □  □  □  □ 

U          |T)L  (  ' 

D.  JrbU  

c.  TIFF  

d.  PDF  

e.     Other  (specify) 

A.  udio 

a.  WAV  

□  □  □  □ 

□  □  □  □ 

□  2 

□  2 

□  2 

□  2 

□  □  □  □ 

U4            L*J            l*J  w 

□  4 

□  4 

□  4 

□  4 

□  ' 
□2 
□3 
□4 

b.  AU  

c.  AIFF  

d.     Other  (specify) 

Video 

a.  MOV  

□  □  □  □ 

□  □  □  □ 

□  □  □  □ 

□  □  □  □ 

□  4 

□  4 

□  4 

□  4 

□  ' 
□2 
□3 
□4 

b.  MPEG  

c.  AVI  

d.     Other  (specify) 

Text 

a.  ASCII  

□  □  □  □ 

□  □  □  □ 

□  2 

□  2 

□  2 

□  2 

□  □  □  □ 

□  4 

□  4 

□  4 

□  4 

□  ' 
□2 
□3 
□4 

b.     Rich  Text  Format  

c.  ANSI  

d.     Other  (specify) 
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8a.  (continued) 


Format 

A.  Formats 
used? 

B.  Standard- 
Is  there... 

C.  Primary  type 
of  format  used 
(Check  one  in 
each  category) 

(Check  all  that 
apply) 

Agency- 
mandated 
standard 

Common 
agency 
practice 

Other 

None 

(Check  one  for  each  category) 

Word  Processing 

a.     Word  Perfect  

□  □.□ 

□3 
□3 
□  3 

□4 
□4 
□  4 

□  ' 

□  2 

□  3 

b.    Microsoft  Word  

c.     Other  (specify) 

Other  (specify) 

□  2 

□  3 

□4 

□  ] 

8b.    If  you  checked  "Other"  category  in  Question  8a,  Column  B,  please  explain. 
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9a.      Is  this  product  in  an  online  medium? 

Yes   □  (Continue  with  question  9b)  No   □  (Skip  to  question  1  Oa) 

9b.      Which  of  the  following  online  approaches  are  used? 

In  Column  A,  indicate  which  type  of  approach  is  used.  (Check  all  that  apply.) 

In  Column  B,  if  the  approach  is  used,  indicate  whether  it  is  mandated  by  the  agency,  a  common  practice  in  the  agency 
although  not  mandated,  other  (i.e.,  not  agency-mandated  standard  or  common  agency  practice,  but  new  and  promising 
and  beginning  to  be  used,  etc.),  OR  none  of  these. 


vanillic  dp|iru<tciics 

A.  Type  of  on- 
line tool  used 

B.  Standard — 
Is  there... 

(Check  all  that 
apply) 

Agency- 
mandated 
standard 

Common 
agency 
practice 

Other 

None 

(Check  one  for  each  category) 

User  Interfaces  Supported 

a.  Netscape  (specify  version) 

b.  Internet  Explorer  (specify  version) 

c.  Telnet  

d.  FTP  

□  □□□□□ 

□  □□□□□ 

□  2 

□  2 

□  2 

1 — 1  0 

□  2 

□  2 

□  2 

□  □□□□□ 

□4 
□4 
□4 

l  1  A 

□  4 

□  4 

□  4 

e.  Nongraphical/dial-up  shell  

f.  Other  (specify) 

Web  Design  Approaches 

a.  Basic  HTML  only  (specify  version) 

b.  Tables  

c.  Frames  

d.  CGI  Scripts  

e.  Use  of  Javascript  

f.  Use  of  Java  Applets  

g.  XML  

i  i 
□  □□□□□□□ 

□  □□□□□□□ 

□2 

□  2 

□  2 

□  2 

□  2 

□  2 

□  2 

□  2 

□  □□□□□□□ 

□  4 

□  4 

□  4 

□  4 

□  4 

□  4 

□  4 

□  4 

h.     Other  (specify) 

Bulletin  Board  Systems  (BBS) 

a.     Graphical  interface/browser  

□  2 

□3 

□  4 

9c.    If  you  checked  "Other"  category  in  Question  9b,  Column  B,  please  explain. 
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Searchability  of  Product 

10a.    Please  indicate  whether  this  product  is...  (Check  all  that  apply.) 


a.  Included  as  part  of  a  full-text  searchable  database  with  no  fielding  

b.  Indexed  by  full-text  and  field   D 1 

c.  Available  as  "view  only"  —  non-searchable   Q 1 

d.  Other  (specify)   □ 1 

10b.    This  product  is  officially  hosted  by...   (Check  all  that  apply.)  (Host  refers  to  the  primary  site  where  the  public  can 
find  the  product.) 


a.  Your  agency   □ 1 

b.  Another  agency  (specify)   D 1 

c.  Contractor   Q 1 

d.  Educational  institution   0 1 

e.  Other  (specify)   D 1 


Retrievability 

1 1 .      This  product  and  any  associated  software. . .  (Check  all  that  apply.) 

a.  Can  be  downloaded,  saved,  and  is  not  subject  to  any  restrictions  on  use  or 

re-use  by  the  end  user   CI 1 

b.  Cannot  be  downloaded,  saved,  and/or  re-used  because  it  is  part  of  a  database 

and  does  not  exist  as  a  distinct  product   \Z\ 1 

c.  Cannot  be  downloaded,  saved,  and/or  re-used  because  it  requires  proprietary 

software  that  is  not  freely  distributable   D1 

d.  Other  (specify)   D1 


C.       PLANNED  PRODUCT  PROFILE  (This  next  section  refers  to  future  plans  for  the  product.) 
Type(s)  of  Data  -  Future  Plans 

12a.     Are  there  any  plans  to  discontinue  publication  of  this  product? 

Yes   □ 1  (Continue  with  question  12b)  No   D2  (Skip  to  question  13a) 

12b.    If  yes,  please  explain.  (Skip  to  Section  D.) 


13a.    What  kind  of  data  will  this  product  contain?  If  product  contains  more  than  one  type  of  data,  respond  for  all  data  types. 
(Check  one.) 


Retain  existing  type(s)  of  data,  no  changes  planned  

.. 

(Skip  to  question  14a) 

Retain  existing  type(s)  of  data  and  add  items  of  one  or  more  new  types  of  data 

(specify) 

□2 

(Continue  with  question  13b) 

Discontinue  one  or  more  types  (specify) 

□3 

(Continue  with  question  13b) 

Change  to  new  type(s)  of  data  (specify) 

□4 

(Continue  with  question  13b) 

;e(s)  will  occur  in  the:  (Check  all  that  apply.) 

(Skip  to  question  14a) 

If  you  checked  both  "short  term"  and"  medium  term"  in  question  13b,  continue  with  question  13c.  Otherwise,  skip  to 
question  13d. 
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13  c.     If  you  have  short-term  and  medium-term  plans,  how  are  your  short-term  plans  different  from  your  medium-term 
plans?  For  example...  (Check  all  that  apply.) 


Short-term  plans  call  for  one  type  of  data,  but  medium-term  plans  call  for  a  different  type  of  data. 

(specify)  ,   □ 1 

In  the  medium  term  there  are  plans  to  combine  resources  to  create  a  new  version  of  a  product  using  a 
different  type  of  data  that  may  be  different  from  short-term  plans. 

(specify)   Q 1 

Other  (specify)   ED 1 


13d.     Do  you  have  any  long-term  plans  (6  or  more  years)  for  changing  this  product  to  new  types  of  data? 

Yes   D 1  (Continue  with  question  lie)  No   ED2  (Skip  to  question  14a) 

1 3e.      If  yes,  please  describe  them  here. 


Medium  -  Future  Plans 

14a.    What  kind  of  medium(s)  will  this  product  use?  If  product  is  delivered  in  more  than  one  medium,  respond  for  all 
mediums.  (Check  one.) 

Retain  existing  medium(s),  no  changes  planned   ED 1  (Skip  to  question  15a) 

Retain  existing  medium(s)  and  add  items  of  one  or  more  new  types  of  mediums 

(specify)   ED   (Continue  with  question  14b) 

Discontinue  one  or  more  types  (specify)   O3  {Continue  with  question  14b) 

Change  to  new  type(s)  of  medium 

(specify)   ED4  (Continue  with  question  14b) 


No  agency-mandated  medium  applies   □  (Continue  with  question  14b) 

14b.    Change(s)  will  occur  in  the:  (Check  all  that  apply.) 

Short  term:  within  1  year  or  less   D1 

Medium  term:  within  2  to  5  years   D 1 

No  changes  indicated   D 1  (Skip  to  question  15a) 

If  you  checked  both  "short  term"  an*/ "medium  term"  in  question  14b,  continue  with  question  14c.  Otherwise,  skip  to 
question  14d. 
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14c.     If  you  have  short-term  and  medium-term  plans,  how  are  your  short-term  plans  different  than  your  medium-term 
plans?  For  example. . .  (Check  all  that  apply.) 


Short-term  plans  call  for  one  type  of  medium,  but  the  medium-term  plans  call  for  a  different  type 

of  medium,  (specify)   O' 

In  the  medium  term  there  are  plans  to  combine  resources  to  create  a  new  version  of  a  product  in  a 
medium  that  may  be  different  from  short-term  plans. 

(specify)   □' 

Other  (specify)   □ 1 


14d.     Do  you  have  any  long-term  plans  (6  or  more  years)  for  changing  mediums  for  this  product? 

Yes   CD1  (Continue  with  question  14e)  No   O2  (Skip  to  question  15a) 

14e.     If  yes,  please  describe  them  here. 


Format  -  Future  Plans 

15a.    What  kind  of  format(s)  will  this  product  contain?  If  the  product  uses  more  than  one  format,  respond  for  all  formats. 
(Check  one.) 

Same  as  existing  format(s),  no  changes  planned   0 1  (Skip  to  question  J 6a) 

Retain  existing  format(s)  and  add  one  or  more  new  format  types 

(specify)   CD2  (Continue  with  question  15  b) 


Change  to  new  format  type(s)  (specify)    D  (Continue  with  question  15b) 


Discontinue  one  or  more  types  {specify)   O4  (Continue  with  question  15b) 

No  agency-mandated  format  applies   □ s  (Skip  to  with  question  16a) 

15b.    Change(s)  will  occur  in  the:  (Check  all  that  apply.) 

Short  term:  within  1  year  or  less   D 1 

Medium  term:  within  2  to  5  years   D 1 

No  changes  indicated   □ 1  (Skip  to  question  16a) 


If  you  checked  both  "short  term"  and  "medium  term"  in  question  15b,  continue  with  question  15c.  Otherwise,  skip  to 
question  15d. 


E-ll 


15c. 


If  you  have  short-term  and  medium-term  plans,  how  are  your  short-term  plans  different  from  your  medium-term 
plans?  For  example...  (Check  all  that  apply.) 


Short-term  plans  call  for  one  kind  of  format,  but  the  medium-term  plans  call  for  a 

different  type  of  format,  (specify)   D ' 

In  the  medium  term  there  are  plans  to  combine  resources  to  create  a  new  version  of  a  product  using  a 
different  format  that  may  be  different  from  short-term  plans. 

(specify)   □ 1 


Other  (specify)  V   □ 1 


15d.     Do  you  have  any  long-term  plans  (6  or  more  years)  for  changing  formats  for  this  product? 

Yes   0'  (Continue  with  question  15e)  No   O2  (Skip  to  question  16a) 

1 5e.     If  yes,  please  describe  them  here. 


D.       OTHER  INFORMATION 
Metadata 

16a.    Is  there  a  metadata  record  for  this  product  (e.g.,  GILS,  MARC)? 

Yes   ED'  (Continue  with  question  16b)         No   O2  (Skip  to  question  17a) 

16b.     If  yes,  please  specify  

Permanent  Access 

17a.     Permanent  public  access  to  this  product  is  currently  provided  by:  (Check  all  that  apply) 

Your  agency   EH1  (Continue  to  question  17b) 

Another  agency  {specify)   O1  (Continue  to  question  17b) 

Other  (specify)   EH 1  (Continue  to  question  17b) 

No  permanent  public  access  provided   D 1  (Skip  to  question  17c) 

17b.     How  is  permanent  public  access  provided?  (specify)  

  (Go  to  question  18a) 

17c.     Are  there  plans  to  provide  permanent  public  access  in  the  future  for  this  product? 

Yes   □'  No   D2 

Permanent  Retention 

1 8a.     Is  this  product  scheduled  for  permanent  retention  by  the  National  Archives  and  Records  Administration? 

Yes   O'  (Continue  with  question  18b)         No   \Z\2  (Skip  to  question  19a) 

18b.     What  is  the  planned  retention  period  for  this  product?  (specify)  
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Ensuring  Authenticity 

19a.     Does  the  agency  ensure  authenticity  (official  status  determination)  for  this  product? 

Yes   (Continue  with  question  19b)         No   Q2  (Skip  to  question  20) 

19b.     How  does  the  agency  attest  to  authenticity?  (specify)  


Updating/Upgrading  Plans 

20.      How  frequently  is  this  product  updated  or  refreshed?  (Check  one.) 

Daily    □' 

Weekly   D2 

Monthly   fj3 

Annually   Q4 

Other  (specify)   □ 5 

21a.    Are  there  plans  for  changing  the  product's  supporting  technology? 

Yes   CD'  (Continue  with  question  2 lb)         No   O2  (Skip  to  question  22a) 

21b.     If  yes,  specify  


User  Fees 

22a.    Is  a  user  fee  charged  for  this  product?  {Check  one. ) 

Yes,  for  all  users   d 1  (Continue  with  question  22b) 

Yes,  for  some  users   d2  {Continue  with  question  22b) 

No   dl3  (Skip  to  question  23a) 

22b.     If  yes,  explain  and  specify  amount  of  fees.   


Licensing 

23a.    Has  the  agency  licensed  commercial  search  and  retrieval  software  for  use  with  this  product? 

Yes   □ 1  (Continue  with  question  23b)  No   D2  (Skip  to  question  24a) 


23b.    Specify  the  software  vendor  and  product  name. 


23c.    Does  the  agency's  license  cover  use  by. . .  (Check  all  that  apply.) 

a.  Agency  personnel   D 1 

b.  Agency's  primary  target  constituencies   CD1 

c.  Federal  Depository  Libraries   D1 

d.  All  libraries   d 1 

e.  Public  users    

f.  Other   
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Public  Domain 

24a.    Is  this  product  in  the  public  domain?  (Check  one.) 


Yes,  for  the  entire  product 
Yes,  for  part  of  product.... 
No  


CH 1  {Continue  with  question  25) 
02  {Continue  with  question  24b) 
D3  (Skip  to  question  24c) 


24b.    If  yes  for  part  of  product,  please  explain.  

24c.    If  no,  has  the  agency  entered  into  an  arrangement  with  the  private  sector  that  would  limit  use  of  this  information? 

(Please  briefly  explain  the  arrangement.) 


E.  COMMENTS 

25.      If  you  wish  to  comment  on  matters  that  you  believe  are  not  otherwise  adequately  covered  in  this  survey,  do  so  here. 


26.      Key  person  completing  this  form. 


Contact  Name: 


Title: 


Telephone: 


Fax: 


E-Mail: 


Other  person(s)  providing  responses  to  questions  or  assistance  in  completing  this  form. 


27. 


Contact  Name: 


Title: 


Telephone: 


Fax: 


E-Mail: 


E-14 


Contact  Name: 


Title:  

Telephone:   Fax: 

E-Mail: 


THANK  YOU.  PLEASE  RETURN  THIS  SURVEY  TO: 

DENISE  GLOVER 
WEST  AT,  ROOM  TA2064 
1650  RESEARCH  BOULEVARD 
ROCKVILLE,  MD  20850 
FAX:  301-517-4134 
PHONE:  301-251-2269 
gloverd  1  @westat.com 

FOR  QUESTIONS  ABOUT  COMPLETING  THE  SURVEY,  CONTACT  DENISE  GLOVER. 
PLEASE  KEEP  A  COPY  OF  THIS  SURVEY  FOR  YOUR  RECORDS. 
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Glossary  of  Terms,  Abbreviations,  and  Acronyms  for  the 
Government  Information  Product  Assessment  Questionnaire 

ASCII — An  acronym  for  American  Standard  Code  for  Information  Exchange,  ASCII  is  an  international  standard 
in  which  numbers,  letters,  punctuation  marks,  symbols,  and  control  codes  are  assigned  numbers  from  0  to  127. 

AIFF — Short  for  Audio  Interchange  File  Format,  a  common  format  for  storing  and  transmitting  sampled  sound. 

ANSI — Acronym  for  the  American  National  Standards  Institute,  a  voluntary  organization  that  creates  standards 
for  the  computer  industry.  In  addition  to  programming  languages,  ANSI  sets  standards  for  a  wide  range  of 
technical  areas,  from  electrical  specifications  to  communications  protocols. 

AU — Short  for  audio,  a  common  format  for  sound  files  on  UNIX  machines. 

AVI — A  format  developed  by  Microsoft  Corporation  for  storing  video  and  audio  information. 

Accessibility — The  degree  to  which  the  public  is  able  to  retrieve  or  obtain  Government  information  products, 
either  through  the  Federal  Depository  Library  Program  (FDLP)  or  directly  through  an  electronic  information 
service  established  and  maintained  by  a  Government  agency  or  its  authorized  agent.  The  other  aspects  of 
accessibility  include  the  degree  to  which  these  Government  information  products  are  available  to  the  public  in  a 
useful  format  or  medium  and  in  a  time  frame  in  which  the  information  has  utility.  In  the  context  of  the  FDLP, 
accessibility  includes  the  degree  to  which  Government  information  is  accurately  identified  and  described 
bibliographically,  the  information's  availability  is  made  known  to  the  public,  and  technological,  social,  economic, 
political  and  physical  barriers  to  gaining  access  are  minimized. 

Authentication — The  certification  of  a  Government  information  product  attesting  to  its  legitimate  official  status. 
Ensuring  the  authenticity  of  a  product  involves  product  design,  planning,  and  policy  development,  as  well  as 
technical  considerations. 

Availability — The  degree  to  which  information  is  physically  or  electronically  obtainable  through  the  intentional 
or  unintentional  provision  of  Government  information  products  to  the  public.  In  the  context  of  the  FDLP, 
availability  includes  the  measures  taken  by  Government  agencies  and  the  FDLP  to  include  Government 
information  products  in  the  program.  Accessibility  is  meaningless  if  information  is  unobtainable  from  its  source. 

BBS — Bulletin  Board  System,  an  electronic  message  center.  Most  bulletin  boards  serve  specific  interest  groups. 
They  allow  users  to  dial  in  with  a  modem,  review  messages  left  by  others,  and  leave  their  own  message. 

CGI  Scripts — Abbreviation  for  Common  Gateway  Interface,  a  specification  for  transferring  information  between 
a  World  Wide  Web  server  and  a  CGI  program.  CGI  programs  are  the  most  common  way  for  Web  servers  to 
interact  dynamically  with  users.  An  increasingly  common  way  to  provide  dynamic  feedback  for  Web  users  is  to 
include  scripts  or  programs  that  run  on  the  user's  machine  rather  than  the  Web  server. 

CD-ROM — Compact  Disk-Read  Only  Memory;  an  optical  disk  from  which  information  may  be  read  but  not 
written. 

DVD — Short  for  digital  versatile  disk  or  digital  video  disk,  a  new  type  of  CD-ROM  that  holds  a  minimum  of 
4.7GB  (gigabytes),  enough  for  a  full-length  movie. 

Digital  Image — An  electronic  version  of  a  bit-mapped  image  of  a  document  or  other  information  format  that 
allows  text  to  be  searched  at  the  character  level;  "digitalize"  means  the  process  and  accompanying  technologies 
required  to  effect  the  conversion  from  bit-mapped  (e.g.,  a  fax)  to  searchable  format. 
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Dissemination — The  act  of  making  Government  information  products  accessible  to  the  public  through 
distribution  to  depository  or  program  libraries  or  by  using  a  Government  electronic  information  service. 

Electronic  Government  Information — Information  that  is  organized,  stored,  and  disseminated  using  electronic 
or  optical  mediums  as  opposed  to  paper-based  or  microfiche-based  mediums. 

FTP — An  acronym  for  of  File  Transfer  Protocol,  the  protocol  (agreed-upon  format)  used  on  the  Internet  for 
sending  files. 

Format — The  manner  in  which  data,  documents,  or  literature  are  organized,  structured,  named  and  described, 
classified,  and  arranged.  For  example,  full  narrative  texts  in  English  language  could  be  in  the  following  forms: 
books  or  articles,  abstracts  of  text  used  in  reviews  and  summaries,  indexes  and  catalogs,  maps,  photographs, 
drawings,  sound  recordings,  video  tapes,  bibliographies,  and  statistical  and  other  numeric  kinds  of  tabulations.  A 
screen  format  is  the  layout  of  fields  on  the  screen.  A  report  format  is  the  layout  of  the  printed  page  including  print 
columns  and  page  headers  and  footers.  A  record  format  is  the  layout  of  fields  within  a  record.  A  file  or  database 
format  is  the  layout  of  fields  and  records  within  a  data  file,  layout  codes  within  a  word  processing  document,  or 
display  lists  (vector)  or  bit  maps  (raster)  within  a  graphics  file.  The  term  is  sometimes  used  to  refer  to  the  way 
digital  data  is  encoded  or  inscribed.  Archivists  used  the  terms  "genre"  or  "form  of  material"  to  cover  what  is 
meant  by  format  in  this  context. 

Frames — A  feature  supported  by  most  modern  Web  browsers  that  enables  the  Web  author  to  divide  the  browser 
display  area  into  two  or  more  sections  (frames).  The  contents  of  each  frame  are  taken  from  a  different  Web  page. 

GIF — Pronounced  jiff  or  giff  (hard  g),  it  is  short  for  graphics  interchange  format,  a  bit-mapped  graphics.  GIF 
supports  color  and  various  resolutions.  It  also  includes  data  compression,  making  it  especially  effective  for 
scanned  photos. 

GILS — Short  for  Government  Information  Locator  Service.  A  metadata  tool  for  identifying,  locating,  and 
describing  publicly  available  Federal  information  resources,  including  electronic  information  resources. 

Gopher — A  method  of  making  menus  of  material  available  over  the  Internet.  Gopher  pre-dates  the  World  Wide 
Web  for  organizing  and  displaying  files  on  Internet  servers. 

Government  Electronic  Information  Service — The  system  or  method  by  which  a  component  of  the 
Government,  or  its  authorized  agent,  disseminates  Government  information  products  to  the  public  via  such  means 
as  a  network  or  use  of  CD-ROMs  at  a  kiosk. 

Government  Information — Refers  to  information,  regardless  of  form,  medium,  or  format,  that  is  created  or 
compiled  by  employees  of  a  component  of  the  Government,  or  at  Government  expense,  or  as  required  by  law. 
Government  information  as  used  here  does  not  include  information  for  official  use  only,  information  classified  for 
reasons  of  national  security,  or  information  used  strictly  for  administrative  or  operational  purposes  (e.g.,  not  of 
public  interest  or  educational  value). 

Government  Information  Product — A  Government  publication  or  other  work  of  the  United  States  Government 
conveyed  in  a  tangible  physical  medium  such  as  a  book  or  CD-ROM,  or  disseminated  through  an  electronic 
Government  information  service  established  and  maintained  by  a  Government  agency  or  its  authorized  agent. 

Graphical  User  Interface — A  program  interface  that  takes  advantage  of  the  computer's  graphics  capabilities  to 
make  the  program  easier  to  use. 

HTML  (HyperText  Markup  Language) — The  authoring  language  used  to  create  documents  on  the  World  Wide 
Web. 
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HTTP — Short  for  HyperText  Transfer  Protocol,  the  underlying  protocol  used  by  the  World  Wide  Web.  HTTP 
defines  how  messages  are  formatted  and  transmitted  and  what  actions  Web  servers  and  browsers  should  take  in 
response  to  various  commands. 

Information  Intermediary — Refers  to  any  person,  institution,  or  mechanism  that  adds  value  to  information 
products  so  that  they  are  more  useful  to  information  users.  Intermediaries  perform  their  work  at  the  middle 
information  life  cycle  stages — that  is,  between  information  creation  and  information  disposition  or  destruction. 
Federal  libraries  and  information  centers  are  examples  of  intermediaries. 

Information  Life  Cycle — The  various  stages  through  which  information  passes,  including  creation,  production  or 
collection,  review  and  editing,  organization  and  reorganization,  packaging,  storage,  search  and  retrieval, 
communication  and  re-communication,  dissemination,  disposition,  archiving,  and  destruction. 

JPEG — Joint  Photographic  Experts  Group  (JPEG)  is  an  image  compression  format  used  to  transfer  color 
photographs  and  images  over  computer  networks. 

Java  Applets — The  use  of  small  Java  programs.  Java  Applets  allows  Web  pages  to  include  functions  such  as 
animations,  calculators,  and  other  fancy  tricks. 

Javascript — A  scripting  language  developed  by  Netscape  to  enable  Web  authors  to  design  interactive  sites. 

MARC — Short  for  machine-reading  cataloguing.  The  USMARC  formats  are  standards  for  the  representation  and 
communication  of  bibliographic  and  related  information  in  machine-readable  form.  The  Library  of  Congress,  in 
consultation  with  various  user  communities,  maintains  USMARC  formats. 

MPEG — Short  for  Moving  Picture  Experts  Group,  a  working  group  of  ISO.  MPEG  generally  produces  high- 
quality  video. 

Medium — The  physical,  chemical,  or  biological  substrate  used  to  create,  organize,  store,  search  for,  retrieve, 
disseminate,  or  permanently  archive  data,  documents,  or  literature  including,  for  example,  paper,  microforms, 
fiber  optic  cables,  photographic  film,  CD-ROM,  floppy  diskettes,  magnetic  storage  devices,  sound  recordings,  and 
videotape. 

Metadata — Metadata  is  data  about  data.  Metadata  describes  how  and  when  and  by  whom  a  particular  set  of  data 
was  collected  and  how  the  data  are  formatted.  These  data  can  be  collections  or  individual  instances  of  objects  or 
documents,  Internet  resources,  etc. 

Migration — The  transfer  of  an  information  product  from  one  hardware  type,  software  package,  system,  network, 
format,  or  medium  to  another.  The  transfer  of  an  information  product  from  a  pre-electronic  medium  such  as  paper 
or  microform  to  an  electronic  medium  such  as  CD-ROM  is  an  example  of  migration. 

PDF — Short  for  Portable  Document  Format.  A  file  format  developed  by  Adobe  Systems,  PDF  captures 
formatting  information  from  a  variety  of  desktop  publishing  applications,  making  it  possible  to  send  formatted 
documents  and  have  them  appear  on  the  recipient's  monitor  or  printer  as  they  were  intended. 

Permanent  Public  Access — The  indefinite,  continuing  accessibility  of  Government  information  products  by  the 
public  including  the  policies,  programs,  formats,  mediums,  and  standards  used.  Simply  because  data  such  as 
weather  and  tide  information,  lists  of  Government  contractors,  editions  of  annual  reports,  and  statistical  series  are 
continuously  updated  with  more  current  information  does  not  necessarily  mean  that  provision  should  not  be  made 
for  retaining  (making  permanently  accessible)  the  replaced  information.  Care  should  be  exercised  in  this  regard, 
however,  to  ensure  no  harm  is  done  to  the  public  by  making  available  out-of-date  information. 

Preferred  Medium  or  Format  Standard — A  medium  or  format  standard  that  may  not  be  agency  mandated  but 
is  either  common  agency  practice  or  applies  to  a  new  or  promising  product  format  or  medium. 
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Public  Domain — A  term  of  American  copyright  law  referring  to  works  that  are  not  copyright  protected;  free  for 
all  to  use  without  permission. 

Refreshing — A  technical  term  meaning  the  manner  in  which  information  mediums  and  technologies  are 
periodically  reassessed  and  upgraded  to  ensure  that  they  are  not  becoming  obsolete,  thereby  risking  the  loss  of 
information  and  the  compromising  of  Permanent  Public  Access  policies.  The  term  is  also  sometimes  applied  to 
formats  and  standards  (not  just  mediums)  that  are  in  danger  of  becoming  obsolete  and  need  to  be  replaced  or 
upgraded. 

Rich  Text  Format  (RTF) — A  standard  for  specifying  formatting  of  documents.  RTF  files  are  actually  ASCII 
files  with  special  commands  to  indicate  formatting  information,  such  as  fonts  and  margins. 

SGML — Short  for  Standard  Generalized  Markup  Language,  SGML  is  a  system  for  organizing  and  tagging 
elements  of  a  document.  SGML  was  developed  and  standardized  by  the  International  Organization  for  Standards 
(ISO)  in  1986. 

Standard — An  agreed-upon  authoritative  convention,  whether  formal  or  informal,  whether  official  or  unofficial, 
whether  de  facto  or  de  jure,  by  which  information  products  are  created,  produced,  formatted,  published,  stored, 
communicated,  and  moved  through  the  remainder  of  the  stages  of  the  information  life  cycle.  Standards  (and  the 
more  informal  concept  and  term  "guidelines")  minimize  incompatibility  and  interoperability  problems  when  an 
information  sender  tries  to  move  data,  documents,  or  literature  into  and  out  of  several  different  formats  and 
mediums  to  a  receiver.  Standards  may  be  set  by  formal  national  or  international  standard- setting  bodies,  or  by 
agencies,  or  by  groups  of  users  informally  by  common  consent. 

TIFF — The  acronym  for  Tagged  Image  File  Format,  a  graphic  file  format  developed  by  Aldus  and  Microsoft. 

Telnet — A  terminal  emulation  program  for  TCP/IP  networks  such  as  the  Internet.  The  Telnet  program  runs  on  a 
user's  computer  and  connects  his/her  PC  to  a  server  on  the  network.  This  enables  users  to  control  the  server  and 
communicate  with  other  servers  on  the  network. 

Type  of  Data — The  general  nature  of  the  information  content  of  a  Government  information  product  and  how  it  is 
arranged,  structured,  and  presented  for  ease  of  handling  in  a  medium.  Some  examples  include  textual  data, 
graphical  data,  spatial  data,  numerical  data,  etc. 

WAIS — Short  for  Wide  Area  Information  Server.  A  distributed  information  service  that  offers  simple  natural 
language  input,  indexed  searching  for  information  retrieval,  and  a  relevance  feedback  mechanism.  It  has  an  easy- 
to-use  interface  that  searches  all  documents  relative  to  your  query,  ranks  them,  and  makes  them  available  to 
retrieve. 

WAV — The  format  for  storing  sound  in  files  developed  jointly  for  Microsoft  and  IBM. 

WORM — Short  for  write  once,  read  many,  an  optical  disk  technology  that  allows  one  to  write  data  onto  a  disk 
just  once. 

World  Wide  Web  (WWW)— WWW  is  a  system  of  Internet  servers  that  support  specially  formatted  documents. 
The  documents  are  formatted  in  a  language  called  HTML  (HyperText  Markup  Language)  that  supports  links  to 
other  documents,  as  well  as  graphics,  audio,  and  video  files. 

XML — Short  for  extensible  Markup  Language.  XML  is  a  pared-down  version  of  SGML,  designed  especially  for 
Web  documents.  It  enables  designers  to  create  their  own  customized  tags  to  provide  functionality  not  available 
with  HTML. 
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Appendix  F 

Site  Visits  to  Three  Federal  Depository  Libraries  and 
Interview  Questions 
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Site  Visits  to  Three  Federal  Depository  Libraries 


Libraries  Visited 


McKeldin  Library 

University  of  Maryland 

College  Park,  Maryland 

(Regional  Federal  Depository  Library) 

Date  visited:  July  30,  1998 


Washington  College  of  Law  Library 
American  University 
Washington,  D.C. 

(Selective  Federal  Depository  Library) 
Date  visited:  August  11,  1998 


Rockville  Regional  Library 
Montgomery  County  Public  Libraries 
Rockville,  Maryland 

(Selective  Federal  Depository  Library) 
Date  visited:  September  9,  1998 


Interviewees 


Linda  Spitzer,  Acting  Head 
Government  Documents  and  Maps 


Joanne  Zich,  Chief 

Government  Documents  and  Media 

Services 
Patrick  Kehoe,  Library  Director 


Judith  Horowitz,  Documents  Coordinator 


Interview  Questions  for  Site  Visits  to  Federal  Depository  Libraries 


Date  of  Site  Visit 

Contact  Name  

Title  

Library  Name  

Address 


Phone   Fax  

Email  

Background  Information 

1 .  How  long  have  you  been  in  your  current  position? 

2.  Describe  range  of  duties  related  to  Government  documents? 

3.  How  long  has  this  institution  been  a  Federal  depository  library? 

4.  Who  are  your  primary  and  secondary  users  of  Government  information  products? 

5.  Do  you  know  what  mediums  patrons  tend  to  be  more  comfortable  using?  (e.g.,  paper,  CD- 
ROMs,  Internet,  diskettes  or  does  it  matter?)  What  about  what  formats  they  prefer  to  use? 
(text  and  graphics) 

Accessibility  for  Users 

6.  What  key  issues  or  concerns  do  users  have  about  accessing  Government  electronic 
information  products?  Have  you  conducted  any  user  surveys  or  focus  groups  to 
determine  the  needs  of  patrons  who  use  Government  information  products  and 

services? 
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If  they  do  not  mention  these  issues,  probe  for  them: 

•  Bibliographic  and  findings  tools  to  identify  and  describe  online  and  electronic  Government 
products 

•  Problems  with  changing  URLs 

•  Charging  user  fees  (specify  types  of  fees:  printing  costs,  fees  to  access  products) 

•  Copyright  restrictions  (can  you  give  an  example?) 

•  Downloading  large  files  from  the  Internet 

•  User  guides  and  documentation 

•  Providing  reference  and  other  public  services  to  people  with  disabilities 

•  Other  (specify) 

Concerns  about  Services  and  Resources 


7.   What  are  your  concerns  about  providing  access  to  electronic  Government  information 
products? 

If  they  do  not  mention,  probe  for: 

•  Are  you  notified  that  a  product  is  available  through  electronic  and  online  storage?  If  yes, 
how? 

•  Are  you  currently  notified  that  a  product  has  been  removed  from  the  Internet?  If  so,  how? 

•  Is  there  a  better  way  to  inform  the  depository  libraries  of  additions,  discontinued  products,  or 
modified  products?  If  so,  what  are  your  recommendations? 

•  What  about  retention  and  preservation  of  Government  electronic  information  products  in  the 
future? 
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(Hardware  and  software  issues  to  probe  for:) 

•  What  do  you  think  about  the  latest  minimum  recommended  specifications  for  public  access 
workstations? 

•  Is  your  library  able  to  keep  pace  with  these  minimum  specifications? 

V 

•  Do  the  current  recommended  minimum  specifications  accommodate  current  online  and 
electronic  Government  information  products? 

•  Other  hardware/software  issues 


Recommendations  for  Improving  Access 


8.  Within  the  last  three  years,  have  you  or  your  predecessor  conducted  any  formal  or 
informal  studies  (i.e.,  surveys,  observations)  of  your  users  to  identify  specific  needs  or 
concerns  they  have  in  using  Government  electronic  information  products? 


9.   If  yes,  could  you  please  briefly  describe  what  you  found  and  what,  if  anything,  you  did 
to  make  adjustments  in  your  services  or  resources? 


10.  If  you  were  given  a  pot  of  money  (say  $25,000)  to  improve  access  to  Government 
electronic  information  products,  how  would  you  spend  the  monies,  in  priority  order? 


11.  If  you  had  unlimited  resources,  what  would  you  do  to  improve  public  access  to  online 
and  electronic  Government  information  products? 


12.  Are  there  other  issues  or  concerns  you  would  like  to  share  with  us  about  the  transition 
to  a  more  electronic  FDLP  that  we  haven't  already  covered? 
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NCLIS  Assessment  of  Electronic  Government  Information  Products 
Summary  Notes  for  Site  Visits  to  Three  Depository  Libraries 


1.    What  mediums  do  patrons  tend  to  be  most  comfortable  using?  (e.g.,  paper,  CD-ROMs, 
Internet,  diskettes)? 


•  Paper  and  microfiche;  Internet  a  close  second. 

•  CD-ROM  is  the  least  preferred  medium  (by  librarians  and  patrons). 


2.   What  key  issues  or  concerns  do  users  have  about  accessing  Government  electronic 
information  products? 


Accessing  Electronic  Resources 

All  three  librarians  expressed  major  concerns  about  the  problems  in  using  Government- 
produced  CD-ROMs  that  are  not  standardized: 

•  The  search  and  retrieval  software  is  different  for  each  CD,  often  they  come  with  no 
installation  instructions  or  user  documentation,  and  they  are  not  user-friendly. 

•  Librarians  must  call  in  a  computer  technician  to  load  the  CDs  and  show  librarians  or 
users  how  to  access  the  information. 

•  Two  of  the  three  libraries  have  significantly  reduced  the  number  of  CD-ROM  titles  they 
select  because  of  the  above-mentioned  problem.  The  third  librarian  indicated  that  they 
have  a  ton  of  CD-ROMs  that  are  not  used  due  to  the  problems  with  loading,  accessing, 
and  using  them. 

Users  are  still  intimidated  by  electronic  mediums  and  computers.  Most  users  ask  librarians 
to  help  them  search  for  materials  on  the  web. 

Since  most  Government  websites  only  contain  the  most  recent  information,  all  librarians 
expressed  concerns  about  users  accessing  retrospective  Government  information  on  the 
web.  Two  selective  depository  librarians  often  send  their  patrons  to  the  regional  depository 
to  use  their  paper  files  for  some  historical  Government  information. 


3.    What  key  library  issues  or  concerns  do  you  have  about  providing  access  to  electronic 
Government  information  products?  


Charging  Fees 

Two  of  the  three  libraries  do  not  currently  charge  fees  for  printing  materials  from  the 
Internet  or  CD-ROMs.  One  library  was  planning  to  change  its  policy  beginning  in 
September  1998  because  it  cannot  continue  to  fund  this  expense.  The  second  library  is  also 
contemplating  changing  its  policy  after  installing  the  next  generation  computer  system. 
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Hardware  issues 

•  Patrons  expect  the  library  to  have  state-of-the-art  equipment  and  expect  librarians  to 
know  how  to  use  it.  Although  all  librarians  felt  their  libraries  had  state-of-the-art 
equipment,  they  were  concerned  about  how  they  could  meet  rising  user  expectations  for 
the  "best"  computer  hardware. 

•  Users  do  not  have  access  to  enough  workstations,  so  the  libraries  must  impose  a  time 
limit  on  computer  use. 

•  Costs  of  hardware,  telecommunications,  and  access  to  electronic  information  have 
increased  dramatically.  For  example,  one  library  reported  spending  $60,000  on  these 
services  and  equipment  10  years  ago;  now  they  spend  about  $270,000  on  hardware 
including  maintenance  agreements. 


Training  Issues 

All  librarians  expressed  concerns  about  finding  time  and  money  to  train  librarians  and 
staff,  especially  on  the  use  of  CD-ROM  products,  but  also  on  downloading  files,  effectively 
searching  the  Internet  for  Government  information,  and  creating  and  maintaining  web 

pages. 


4.   If  you  were  given  a  pot  of  money  (e.g.,  $25,000)  to  improve  access  to  Government  electronic 
information  products,  how  would  you  spend  the  monies,  in  priority  order? 


•  New  CD-ROM  server;  an  18-disk  CD  changer  (3  responses). 

•  More  staff  training  (2  responses). 


5.    If  you  had  unlimited  resources,  what  would  you  do  to  improve  public  access  to  online  and 
electronic  Government  information  products? 


•  Form  partnerships  with  GPO  and  a  Government  agency  to  put  some  retrospective 
Government  information  on  their  server  so  it  will  be  accessible  to  users  in  the  future  (2 
responses). 

•  Provide  outreach  to  public  schools,  community  centers,  etc.,  to  educate  students  and 
adults  about  the  wide  variety  of  valuable  information  available  from  the  Federal 
Government  (2  responses). 

Other  Comments: 

One  librarian  expressed  strong  feelings  about  the  need  for  Congress  to  provide  long-term 
financial  support  to  Federal  depository  libraries  so  they  can  provide  permanent  public 
access  to  digital  media. 
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Appendix  G 

Electronic  Government  Information  Products  Assessment 
Agency  Meetings  Held  and  Discussion  Questions 
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Electronic  Government  Information  Products  Assessment 
Agency  Meetings  Held 


Agency  Number  of  Attendees 


Department  of  Health  and  Human  Services  1 0 

Meeting  Date:  September  14,  1998 


Supreme  Court  of  the  United  States  1 1 

Meeting  Date:  September  15,  1998 


Environmental  Protection  Agency  12 
Meeting  Date:  September  17,  1998 


Department  of  Education 
Meeting  Date:  September  22,  1998 


Department  of  Commerce 
Meeting  Date:  September  23,  1998 


National  Archives  and  Records  Administration 
Meeting  Date:  September  24,  1998 
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Electronic  Government  Information  Products  Assessment 
Agency  Meeting  Discussion  Questions 

1.  Does  your  agency  have  preferred  medium  and  format  standards  for  Government  electronic 
information  products  at  the  permanent  public  accessibility  stage?  If  so,  what  are  the  top 
three?  What  factors  does  your  agency  consider  in  determining  preferred  standards  (e.g.,  user 
needs,  agency's  dissemination  requirements  or  policies,  cost,  security  etc.)?  What  about 
specific  standards  for  CD-ROMs  as  they  relate  to  user  documentation,  installation,  search 
software,  etc.? 

2.  Can  you  give  us  any  examples  of  particularly  innovative  and  creative  product  formats, 
mediums  and/or  online  approaches?  We  have  in  mind  formats,  mediums,  or  online 
approaches  that  may  well  point  to  the  wave  of  the  future,  not  only  for  a  particular  product  but 
also  for  other  kinds  of  products,  yet  is  neither  an  agency-mandated  standard  nor  even  a 
common  agency  practice. 

3.  Is  there  any  difference  between  your  agency's  preferences  for  mediums  and  formats  as 
opposed  to  the  preferences  of  intermediary  distributors?  If  so,  what  are  those  differences,  and 
why  are  the  two  preferences  different? 

4.  Has  your  agency  involved  external  user  groups  in  assessing  the  value  and  effectiveness  of  the 
dissemination  of  Government  electronic  information  products?  If  so,  are  there  formats  and 
mediums  that  seem  particularly  appropriate  for  public  dissemination  to  users  who  may  be 
economically,  technically,  or  physically  disadvantaged? 

5.  Does  your  agency  follow  any  internally  or  externally  prescribed  guidelines  for  the 
presentation  and  organization  of  products  in  online  formats?  If  so,  what  are  they  (e.g., 
WWW  Federal  Consortium,  FIPS  Guidelines,  agency  or  departmental  publication 
specifications  or  guidelines)? 

6.  Has  your  agency  undertaken  any  kind  of  cost  benefit  analyses  for  producing  or  creating 
products  in  preferred  or  emerging  formats,  mediums,  or  online  approaches  for  distribution  to 
the  FDLP?  If  so,  which  ones  appear  to  be  the  most  cost-effective? 

7.  What  factors  does  your  agency  consider  in  deciding  to  create  or  retain  products  in  more  than 
one  medium?  Is  this  a  common  agency  practice? 
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8.  Are  there  trends  with  respect  to  migrating  specific  families  of  products  from  pre-electronic 
mediums  to  electronic  mediums  or  formats?  For  example,  are  loose-leaf  publications, 
training  manuals,  annual  reports,  conference  proceedings,  newsletters,  rules  and  regulations, 
scientific  journals,  etc.,  targeted  for  migration  to  a  particular  medium?  If  so,  which  mediums 
and  formats  are  used  for  specific  families  of  products? 

9.  Has  your  agency  identified  any  medium  and  format  standards  that  seem  particularly 
appropriate  for  use  throughout  a  product's  entire  information  life  cycle,  not  just  at  one  stage 
(i.e.,  creation,  storage  and  retrieval,  communication  and  dissemination,  archiving  and 
disposition)  for  electronic  Government  information  products?  If  so,  which  ones? 

10.  How  do  you  determine  whether  a  product  should  be  made  permanently  publicly  accessible 
when  you  create  or  produce  it?  If  so,  what  criteria  do  you  use  to  determine  which  products 
will  be  permanently  publicly  accessible?  Can  you  give  us  any  examples  of  how  you  ensure 
permanent  public  accessibility  for  a  given  product? 

1 1 .  Does  your  agency  routinely  provide  locator  tools  (e.g.,  GILS  or  specific  agency  locators)  to 
enhance  access  to  information  sources  and  services  available  to  external  users  and  customers? 
If  so,  is  this  an  official  policy,  common  agency  practice,  or  both? 

12.  Are  there  trends  for  facilitating  public  access  to  your  agency  products  by  including  them  in 
broad  electronic  Federal  Government  information  services  such  as  GPO  Access,  LOC 
Thomas,  and  NTIS  FedWorld?  Are  you  using  any  particular  guidelines  to  facilitate  that 
decision,  and  if  so,  what  are  they? 
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Summary  of  Responses  to  Agency  Meeting  Questions 


1.  Does  your  agency  have  preferred  medium  and  format  standards  for  Government  electronic 
information  products  at  the  permanent  public  accessibility  stage?  If  so,  what  are  the  top 
three?  What  factors  does  your  agency  consider  in  determining  preferred  standards. 


Agencies  reported  using  the  following  electronic  mediums  most  often:  CD-ROM,  Internet, 
and  Bulletin  Board  System  (BBS).  The  most  frequently  used  formats  include: 

•  HTML,  PDF,  ASCH 

•  TD7F,  GD7,  JPEG 

•  Lotus/Domino 

In  determining  medium  and  format  standards,  agencies  consider  the  amount  of  information 
or  files  used,  the  timeliness  of  the  information  (e.g.,  more  recent  products  or  publications 
often  placed  on  the  web),  and  user  needs  for  easy  and  quick  access  to  information.  For 
example,  some  agencies  are  looking  to  Windows  as  an  interface  for  CD-ROM  products 
since  the  public  is  used  to  seeing  and  using  Windows. 


2.  Do  you  have  any  examples  of  particularly  innovative  and  creative  product  formats,  mediums, 
and/or  online  approaches?  We  have  in  mind  formats,  mediums  or  online  approaches  that  may 
well  point  to  the  wave  of  the  future  for  not  only  a  particular  product  but  also  for  other  kinds 
of  products,  yet  is  neither  an  agency-mandated  standard  nor  even  a  common  agency  practice. 


Almost  all  the  agencies  interviewed  are  exploring  a  wide  range  of  innovative  and  creative 
format,  medium,  and  web  applications.  Below  is  a  sampling  of  some  of  the  interesting 
online  approaches  and  formats  used  by  the  agencies  interviewed. 

•  Data  warehousing:  Taking  information  not  previously  publicly  accessible  and 
integrating  it  into  an  online  format.  The  format  used  is  an  Oracle  database  using  SQL 
to  query. 

•  Interactive  Geographic  Information  System  (GIS).  Provide  mapping  capability  through 
GIS  combined  with  regulatory  information  to  create  dynamic  maps. 

•  Online  catalog  of  all  products  on  the  agency's  website,  using  Oracle  with  a  ColdFusion 
interface  with  their  search  engine  (Verity). 

•  Creating  user  guides  for  CD-ROMs  as  pop-up  HELP  or  short  Read-Me  files  so  users 
will  be  more  inclined  to  use  HELP. 

•  Live  "real-time"  web  casting  of  selected  speeches.  Format:  RealPlayer  software, 
available  free  from  the  web. 

•  Radio  news  broadcast  news  service.  Provides  daily  radio  sound-bits  for  news  reports. 
Format:  RealAudio  RealPlayer  software  and  WAV  file  format  for  downloading. 
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•  Real-time  forecasting  of  air  pollution  levels  for  22  states  from  one  site.  Format: 
animated  GIFs  created  by  nonproprietary  software  designed  by  computer  center  in 
North  Carolina. 

•  Multimedia  CD-ROM  (i.e.,  art,  music,  animation,  fdm,  and  video). 

•  Searchable  electronic  inventory  of  all  proposals  funded  over  the  last  30  years  so  the 
agency  can  analyze  its  own  information  and  make  it  available  to  others.  Format:  legacy 
database  put  into  WAV  database  using  HTML  on  the  fly. 

•  Documents  stored  in  TIFF  format  for  image  and  textual  data.  As  customers  request 
documents,  the  agency  converts  them  to  PDF  so  customer  can  download. 


3.    Is  there  any  difference  between  your  agency's  preferences  for  mediums  and  formats  as 
opposed  to  the  preferences  of  intermediary  distributors? 


Generally,  agencies  indicated  that  intermediary  distributors  do  not  find  agency  formats  to 
be  restrictive.  However,  the  distributors  often  modify  formats  (e.g.,  from  HTML  to  ASCII, 
or  reformat  data  using  compression  technology). 


4.  Has  your  agency  involved  external  user  groups  in  assessing  the  value  and  effectiveness  of  the 
dissemination  of  electronic  Government  information  products?  If  so,  are  there  formats  and 
mediums  that  seem  particularly  appropriate  for  public  dissemination  to  users  who  may  be 
economically,  technically,  or  physically  disadvantaged? 


Yes,  all  agencies  reported  that  they  involve  users  in  assessing  some  aspects  of  their 
products,  per  OMB  Circular  A-130.  Examples  include  the  following: 

Focus  groups  are  used  to: 

•  Determine  the  capability  of  new  products  (e.g.,  Can  you  use  the  same  technology  for 
the  newer  version  of  a  product?  Do  users  lose  anything  (e.g.,  Macros)  when  they 
update  a  product?). 

•  Determine  ways  to  create  more  user-friendly  CD-ROMs  that  resulted  in  the  agency 
establishing  three  principles  for  producing  CD-ROMs:  make  them  simple  to  use, 
intuitive,  and  self-tutorial. 

•  Determine  how  information  is  presented  on  the  web  (Alpha  and  Beta  testing). 

•  Solicit  feedback  on  usability  and  accessibility;  focus  groups  conducted  with  tribal 
Governments,  teachers,  librarians,  children,  etc. 

•  Learn  about  expectations,  behavior,  and  problems  in  accessing  products  on  the  web 
experienced  by  the  elderly. 

•  Interview  and  videotape  users  to  assist  agency  redesign  of  website. 
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•    Solicit  ideas  from  educators  at  professional  conferences  about  topics  that  they  would 
like  to  see  presented  in  online  products. 

For  technologically  or  economically  disadvantaged  customers,  one  agency  sub-unit  faxes 
free  copies  of  information  printed  from  the  CD-ROM  or  the  Internet.  Another  agency  sub- 
unit  reported  they  try  to  reach  economically  disadvantaged  customers  by  training  rural 
community  leaders  in  isolated  areas  on  ways  to  access  health-related  information  on  the 
web  . 


5.  Does  your  agency  follow  any  internally  or  externally  prescribed  guidelines  for  the 
presentation  and  organization  of  products  in  online  formats?  If  so,  what  are  they  (e.g., 
WWW  Federal  Consortium,  FIPS  Guidelines,  agency  or  departmental  publication 
specifications  or  guidelines)? 


Most  agencies  have  developed  guidelines  or  "best  practices"  for  presentation  of  products  in 
online  formats.  However,  several  agency  representatives  indicated  that  the  real  challenge  is 
in  convincing  agency  staff  to  comply  with  the  guidelines.  The  following  agencies  provided 
information  on  their  guidelines: 

1.  NARA:  NARA  Guidelines  for  Digitizing  Archival  Materials  for  Electronic  Access  (not 
to  be  considered  a  standard  for  digital  imaging). 

(http://www.nara.gov/nara/vision/eap/eapspec.html) 

2.  Federal  Web  Consortium's  guidelines  are  based  on  Dept.  of  Education's  guidelines 

(http://www.ed.gov/internal/wwwstds.html).  One  sub-unit,  NCES,  also  has  developed 
guidelines. 

3.  EPA  used  WWW  Federal  Consortium  guidelines 

(http://www.dtic.mil/staff/cthomps/guidelines/)  to  develop  their  own  guidelines  for 
presentation. 

4.  Census  uses  a  process  and  structure  for  submitting  items  for  the  web,  but  it  is  not  yet 
formalized. 

5.  No  departmental  guidelines  exist  for  the  fifth  agency,  but  most  sub-units  have  some 
kind  of  guidelines  for  presentation  and  organization,  although  they  might  vary  among 
sub-units. 
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6.  Has  your  agency  undertaken  any  kind  of  cost-benefit  analyses  for  producing  or  creating 
products  in  preferred  or  emerging  formats,  mediums,  or  online  approaches  for  distribution  to 
the  FDLP?  If  so,  which  ones  appear  to  be  the  most  cost-effective? 


Generally,  agencies  have  not  conducted  a  formal  cost-benefit  analysis.  Agency 
representatives  made  the  following  observations,  however: 

•  One  agency  sub-unit  tracks  the  number  of  customers  who  purchase  a  product  and 
compares  this  amount  against  the  cost  of  producing  it.  They  discontinued  a  product  on 
CD-ROM  because  so  few  people  could  afford  to  buy  it. 

•  The  web  reduces  administrative  costs  for  printing  and  mailing  hard  copies  of 
publications. 

•  One  agency  reported  a  dramatic  decrease  (by  25,000)  in  the  number  of  publications 
requested  under  the  Freedom  of  Information  Act  due  to  the  web. 

•  One  agency  tracks  the  number  of  people  that  a  product  can  potentially  reach  (e.g.,  they 
sell  2,000  paper  copies  through  GPO,  but  have  9,000  hits  on  the  website). 

•  One  agency  reported  that  they  order  fewer  publications  to  fill  customer  requests  as  a 
result  of  the  web. 


7.    What  factors  does  your  agency  consider  in  deciding  to  create  or  retain  products  in  more  than 
one  medium?  Is  this  a  common  agency  practice? 


Agency  representatives  reported  that  they  consider  a  variety  of  factors  in  creating  and 
retaining  products  in  more  than  one  medium,  although  they  did  not  characterize  these 
factors  as  a  common  agency  practice.  Several  agencies  reported  that  these  issues  are 
considered  on  a  case-by-case  basis  or  by  the  individual  program  unit.  The  key  factors 
considered  are: 

1.  Budget  (e.g.,  some  products  in  CD-ROM  are  too  expensive  to  make  available  to  a  small 
audience). 

2.  Cost  (e.g.,  cost  to  print  and  mail  product  as  opposed  to  make  it  available  on  the  web). 

3.  Needs  of  technologically  disadvantaged  users  (e.g.,  one  agency  maintains  its  Fax  on 
Demand  service,  even  though  it  is  not  cost-effective). 

4.  Accessibility  (e.g.,  one  sub-unit  stores  products  in  TIFF  image  format  so  they  can 
produce  them  in  whatever  medium  of  output  customers  want). 

5.  Size  of  audience  (e.g.,  agencies  survey  users  and  use  web  software  to  track  use). 

6.  Number  of  queries  or  type  of  customer  requests.    Customers  will  often  request  a 
publication  or  product  in  more  than  one  medium. 
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7.  Regulatory  requirements.    Some  products  must  be  produced  in  paper  regardless  of 
whatever  other  medium  it  is  produced  in. 

8.  Type  of  product/publication  (e.g.,  searchable  databases  are  only  suitable  for  electronic 
mediums). 


8.  Are  there  trends  with  respect  to  migrating  specific  families  of  products  (e.g.,  loose-leaf 
publications,  training  manuals,  annual  reports,  conference  proceedings)  from  pre-electronic 
mediums  to  electronic  mediums  or  formats?  If  so,  which  mediums  and  formats  are  used  for 
specific  families  of  products? 


The  general  trend  reported  by  agency  representatives  is  to  migrate  more  products  to  the 
web,  especially  recent  ones.  Some  examples  include: 

•  Conference  proceedings  and  presentations  online  in  PowerPoint  or  PDF. 

•  Newsletters  in  HTML. 

•  Training  manuals  and  annual  reports  in  HTML  and  PDF. 

•  Information  for  records  managers  are  posted  to  Gopher,  but  will  move  to  the 
agency's  website  in  1999. 


9.  Has  your  agency  identified  any  medium  and  format  standards  that  seem  particularly 
appropriate  for  use  throughout  a  product's  entire  information  life  cycle,  not  just  at  one  stage 
(i.e.,  creation,  storage  and  retrieval,  communication  and  dissemination,  archiving  and 
disposition)  for  electronic  Government  information  products?  If  so,  which  ones? 


Most  agencies  have  either  not  addressed  this  issue  of  information  life  cycle  or  are  struggling 
with  it. 

•  NARA  has  established  medium  and  format  standards  for  transferring  permanent 
records  to  the  National  Archives  in  36CFR  1228.188. 

•  One  sub-unit  reporting  putting  documents  in  ASCII,  but  using  Oracle  for  database 
management. 

•  One  sub-unit  is  beginning  to  think  about  standardization  for  some  documents.  They 
draft  documents  in  Lotus  Notes  (GroupWare)  and  publish  final  document  in 
another  database  that  goes  onto  the  web.  They  use  Rich  Text  Format  (RTF)  to 
accommodate  images  and  text. 

•  Several  sub-units  indicated  that  the  technology  is  changing  so  rapidly  they  cannot 
establish  standards. 
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10.  How  do  you  determine  whether  a  product  should  be  made  permanently  publicly  accessible 
when  you  create  or  produce  it?  If  so,  what  criteria  do  you  use  to  determine  which  products 
will  be  permanently  publicly  accessible?  Can  you  give  us  any  examples  of  how  you  ensure 
permanent  public  accessibility  for  a  given  product? 


No  agencies  could  provide  responses  to  this  question  or  indicated  that  this  issue  has  not 
been  resolved.  Some  observations: 

•  One  sub-unit  is  committed  to  making  paper  and  CD-ROM-based  products  available 
for  permanent  access,  but  they  are  less  clear  about  their  commitment  to  products  on 
the  Internet. 

•  Some  agency  representatives  did  not  understand  the  differences  between  permanent 
public  access  and  permanent  records. 

(The  experts  interviewed  for  this  study  provide  some  insight  into  the  reasons  that  agencies 
are  not  addressing  this  issue.) 


11.  Does  your  agency  routinely  provide  locator  tools  (e.g.,  GILS  or  specific  agency  locators)  to 
enhance  access  to  information  sources  and  services  available  to  external  users  and  customers? 
If  so,  is  this  an  official  policy,  common  agency  practice,  or  both? 


Most  agencies  indicated  that  the  web  format  supercedes  the  original  GILS  concept. 
However,  most  agencies  have  their  own  locators: 

•  NTIS  has  a  catalog  and  maintains  some  GILS  records. 

•  EPA's  website  has  a  GILS  record  and  they  put  all  Internet  products  on  one  server  so 
there  is  one  access  point  for  all  their  products. 

•  Development  and  maintenance  of  GILS  records  is  official  agency  policy  for  NARA. 


12.  Are  there  trends  for  facilitating  public  access  to  your  agency  products  by  including  them  in 
broad  electronic  Federal  Government  information  services  such  as  GPO  Access,  LOC 
Thomas,  and  NTIS  FedWorld?  Are  you  using  any  particular  guidelines  to  facilitate  that 
decision,  and,  if  so,  what  are  they? 


About  half  of  the  agencies  use  GPO  Access  or  NTIS  FedWorld.  The  other  half  relies  more 
heavily  on  individual  agency  websites  with  good  links. 
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Appendix  H 

Assessment  of  Electronic  Government  Information  Products 
List  of  Expert  Interviews  and  Interview  Questions 
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Assessment  of  Electronic  Government  Information  Products 
List  of  Expert  Interviews  and  Discussion  Questions 

Interviewees  Date  of  Telephone  Interview 

Jerry  Malitz,  Webmaster  October  27,  1998 

National  Center  for  Education  Statistics 
U.S.  Department  of  Education 
Washington,  D.C. 

Linda  Wallace,  Chief  October  27,  1998 

Electronic  Information  Services 
Internal  Revenue  Service 
Washington,  D.C. 

Evelyn  Frangakis,  Preservation  Officer  November  10,  1998 

National  Agricultural  Library 
U.S.  Department  of  Agriculture 
Beltsville,  Maryland 

Abby  Smith,  Director  of  Programs  November  10,  1998 

Council  on  Library  and  Information  Resources 
Washington,  D.C. 

John  Bertot,  Associate  Professor  November  18,  1998 

State  University  of  New  York  at  Albany 
Albany,  New  York 

Charles  McClure,  Distinguished  Professor,  November  24,  1998 

School  of  Information  Studies 
Syracuse  University 
Syracuse,  New  York 
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Interview  Questions  for  Webmasters: 
Jerry  Malitz  and  Linda  Wallace 
(October  27,  1998) 

Role  of  Webmaster 

1.  How  long  have  you  been  in  your  current  position  as  webmaster?  When  and  how  was  the  position 
created?  Were  you  the  first  webmasteV  in  your  agency?  How  does  the  position  reside 
administratively  in  the  structure  of  your  agency?  What  office  or  unit  do  you  report  to? 

2.  Please  describe  your  current  job  responsibilities  and  duties.  What  portion  of  the  following  skills, 
experience,  and  knowledge  do  you  use  to  perform  your  job:  technical,  administrative,  analytical, 
program,  other? 

3 .  Is  there  a  formal  or  informal  structure  for  working  with  staff  and  administrators  in  other  departments 
or  units  (e.g.,  program  staff,  IT,  publications,  public  relations,  records  managers,  librarians,  etc.)?  If 
yes,  please  describe  how  you  interact  with  them? 

4.  What  do  you  envision  as  the  future  role  of  the  webmaster  in  Federal  Government  agencies?  Do  you 
see  your  role  as  being  very  different  in  5  years  than  it  is  now?  How? 

Format  Standards  and  Public  Accessibility 

5.  Please  describe  the  website  development  process  in  your  agency  from  the  time  you  receive  or 
generate  requests  through  design,  development,  evaluation,  testing,  and  implementation,  etc. 

6.  Has  your  agency  developed  policies  or  guidelines  including  format  standards  to  ensure  technical 
consistency  in  the  development  of  web  products  that  are  intended  for  public  dissemination?  What  are 
the  most  frequently  used  file  formats  and  why?  Can  you  identify  any  formats  you  plan  to  use  in  the 
future? 


7.  Are  there  limitations  or  specific  designations  of  software  tools  that  may  be  used  to  develop  and 
implement  web  pages  or  sites?  What  standards  are  applied  to  configuration  control  and  arrangement 
of  web-based  applications?  Do  you  have  a  direct  role  in  determining  these  standards,  or  are  they 
developed  at  an  agency  or  departmental  level? 

8.  Are  there  general  security  standards  applied  to  the  availability  or  distribution  of  web-based 
information?  What  is  your  role  in  determining  or  implementing  these  standards?  Which  software 
products  do  you  use  to  implement  these  standards? 
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9.  How  do  you  evaluate  the  effectiveness  of  your  websites?  Who  is  involved  in  the  process?  What 
methodology  and  criteria  have  you  or  others  used  to  evaluate  websites? 

10.  Has  your  agency  discussed  the  concept  of  permanent  public  accessibility  as  it  relates  to  Government 
electronic  information  products  intended  for  public  dissemination?  How  is  your  agency  addressing 
the  concerns  of  librarians,  GPO,  and  others  for  ensuring  permanent  public  accessibility  for  electronic 
Government  information  products? 

1 1 .  What  consideration  are  you  giving  to  creating  a  metadata  record  for  your  information  resources  or 
services  on  the  web  (e.g.,  GILS,  MARC,  or  specific  agency  locators)? 

Cost  Analysis 

12.  Linda,  we  know  you  have  collected  data  on  the  comparative  costs  of  delivering  services  to  customers 
via  different  delivery  mechanisms  such  as  mail,  e-mail,  Fax  on  Demand,  kiosks,  Internet,  telephone, 
walk-in,  CD-ROM,  etc.  What  have  you  learned  about  the  costs  of  delivering  services  to  customers 
using  these  different  systems?  Which  delivery  mechanisms  are  the  most  cost-effective  for  what  types 
of  services? 
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NCLIS  Assessment  of  Electronic  Government  Information  Products 
Summary  Notes  from  Interview  with  Linda  Wallace  (IRS)  and  Jerry  Malitz  (NCES) 


1 .    Please  describe  your  current  job  responsibilities  and  duties. 


WALLACE  (IRS):  \ 

(Wallace  was  a  telecommunications  expert  and  technical  advisor  to  CIO  when  IRS  asked  her  to  be 
webmaster.  She  also  holds  the  title  of  Chief,  Electronic  Information  Services.  She  is  responsible  for 
all  electronic  information  products  including  Fax  on  Demand,  Internet,  e-mail,  etc.) 

Wallace's  three  major  areas  are  content,  applications,  and  development.  Her  office: 

•  Generates  new  services  and  authorware,  including  creating  automated  filters  and  templates 
through  core  knowledge  repository. 

•  Participated  in  the  development  of  SGML  format  (standard  format  for  IRS  since  1970s). 

•  Interacts  with  customers  to  automate  a  standard  way  to  build  a  core  knowledge  repository. 
Repository  contains  automated  templates  and  filters  to  generate  media  output  to  serve 
customers  via  Internet,  Fax  on  Demand,  CD-ROM,  bulletin  board  system,  telephone,  or  mail 
requests. 

a.  Core  repository  can  satisfy  95  percent  of  requests  using  86  different  variables  or 
attributes  that  are  indexed  so  everything  is  searchable.  All  documents  include  individual 
catalog  and  document  numbers. 

b.  Documents  are  always  authored  in  SGML  and  can  automatically  be  converted  into  a 
different  format  or  posted  on  the  web,  BBS,  etc.,  in  10-12  hours  to  fill  customer 
requests. 

c.  Filters  and  templates  are  solely  by  this  group.  They  also  track  history  of  a  document. 

d.  A  knowledge  base  is  being  built  by  developing  a  database  of  frequently  asked  questions. 

e.  They  use  ICON  tagging  to  provide  accessibility  to  the  visually  impaired.  All  DiS 
documents  are  ADA-compliant,  online  searchable,  and  downloadable. 

MALITZ  (NCES): 

As  Technology  Outreach  Officer  for  NCES,  Malitz  services  state  education  agencies,  school  districts,  etc. 

a.  Each  one  of  NCES's  30  programs  has  a  web  publisher  and  a  web  liaison. 

b.  Contractors  actually  prepare  materials  for  the  web  once  program  officer  has  approved  content. 

c.  Malitz  sets  standards,  guidelines,  and  procedures  for  web  publisher  to  follow. 
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d.  Web  publisher  in  program  area  develops  website  on  a  separate  development  server;  Malitz 
reviews  and  makes  technical  changes  to  ensure  that  the  site  meets  minimum  standards  and 
guidelines. 

e.  Sometimes,  he  develops  a  new  application  for  others  to  use  (e.g.,  NEWS  FLASH  subscription 
service  featuring  daily  breaking  news  from  the  Department  of  Education). 


2.  Do  you  have  a  formal  or  informal  structure  for  working  with  staff  and  administrators  in  other 
departments  or  units  (e.g.,  program  staff,  IT,  publications,  public  relations,  records  managers, 
librarians,  etc.)?  If  yes,  please  describe  how  you  interact  with  them. 


WALLACE: 

Wallace  deals  with  high-level  senior  executives,  review  ing  their  business  plans,  problems,  and  goals, 
then  recommending  solutions  that  include  productivity  measures,  production  rates,  and  cost  per 
person.  In  one  case,  she  recommended  a  business  CD-ROM. 

•  Establishes  strong  liaisons  with  industry  (has  marketing  person  on  staff).  They  receive  one-half 
of  funding  from  industry  to  support  business  projects  that  benefit  industry  and  IRS  customers. 

•  Established  various  delivery  service  programs:  Internet  in  '96,  Fax  on  Demand  in  '96,  CD- 
ROM  in  '95,  BBS  a  while  ago. 

MALITZ 

•  Each  one  of  NCES's  30  programs  has  a  web  publisher  and  a  web  liaison. 

•  Contractors  prepare  materials  for  the  web  once  content  has  been  approved  by  the  program 
officer. 

•  Web  publisher  in  program  area  develops  web  site  on  a  separate  development  server;  when 
finished,  Malitz  reviews  and  makes  technical  changes  to  ensure  that  site  meets  minimum 
standards  and  guidelines. 

•  Sometimes,  Malitz  develops  a  new  application  for  others  to  use  such  as  the  NEWS  FLASH 
subscription  service  that  features  daily  breaking  news  from  the  Department  of  Education. 

•  Malitz's  work  is  divided  fairly  evenly  between  technical,  administrative,  analytical,  and 
program  areas. 
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3.    What  do  you  envision  as  the  future  role  of  the  webmaster  in  Federal  Government  agencies?  How  you 
see  your  role  as  in  5  years  as  compared  to  now? 


MALITZ: 

The  role  of  webmaster  will  be  completely  different.  In  the  future,  he/she  will  have  more  of  a 
coordinating  function  and  will  set  policies  and  procedures.  The  program  staff  will  be  forced  to  do 
their  own  work  on  the  web,  just  as  they  now  do  their  own  word  processing  and  e-mail. 

WALLACE: 

The  role  of  the  webmaster  will  be  that  of  an  enabler  for  business  units  with  everyone  involved. 
There  will  be  more  of  a  focus  on  multimedia  (e.g.,  BBS,  CD-ROM,  Fax  on  Demand)  and  not  just  the 
Internet. 


4.   Please  describe  the  website  development  process  in  your  agency  from  the  time  you  receive  or 
generate  requests  through  design,  development,  evaluation,  testing,  and  implementation,  etc. 


WALLACE: 

Her  unit  receives  and  generates  requests.  Requests  from  the  core  repository  can  fit  into  an  existing 
template  filter  application. 

a.  Staff  and  contractors  conduct  testing  and  implementation. 

b.  Second  year  after  web  in  operation,  requests  for  paper  copies  of  forms  dropped  by  50 
percent. 

c.  Provide  hidden  codes  to  track  where  returns  come  from:  fax,  Internet,  libraries,  phone 
requests,  etc. 

d.  Evaluation  includes  a  simple  three-question  customer  service  survey  on  content:  did  you  get 
what  you  needed,  where  would  you  have  gone  if  not  here?  They  build  evaluation  into  every 
step  of  the  process. 

e.  A  panel  of  experts  measures  the  effectiveness  of  their  websites.  Also,  they  have  partnered 
with  schools  to  recruit  instructors  and  students  to  review  site  before  they  go  "live." 

f.  One  person  reviews  all  e-mail  messages  that  contain  feedback  on  website;  use  automated 
sorters  by  key  words  to  batch  the  type  of  feedback  received. 
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MALITZ: 


Before  website,  customers  were  very  specialized.  Most  were  data  file  users.  After  they  created  their 
website,  their  customer  based  increased  tremendously.  Now  the  culture  is  different  and  NCES  is 
dealing  with  questions  from  the  general  public. 

a.  NCES  uses  a  developmental  server  but  is  planning  to  implement  a  Point-to-Point  Tunnel 
Protocol  (PPTP)  so  the  developmental  server  is  behind  the  firewall  and  no  longer  open  to 
everyone.  Only  web  publishers  and  contractors  will  have  access  to  server. 

b.  Malitz  never  reviews  content;  that  is  done  by  individual  program  staff. 

c.  NCES  conducts  customer  surveys  of  users  to  develop  and  refine  sites. 

d.  Malitz  does  database  development  and  tests  multiple  browsers.  NCES  is  UNLY-based;  rest 
of  ED  is  Windows-based. 


5.   Has  your  agency  developed  policies  or  guidelines  including  format  standards  to  ensure  technical 
consistency  in  the  development  of  web  products  that  are  intended  for  public  dissemination?  What  are 
the  most  frequently  used  file  formats  and  why?  Can  you  identify  any  formats  you  plan  to  use  in  the 
future? 


WALLACE: 

Formats  most  frequently  used  are  SGML,  PDF,  HTML,  and  Postscript,  respectively.  They  will  add 
XML  soon.  They  train  authors  to  use  SGML.  SGML  is  "intelligent"  data  that  can  automatically 
generate  other  formats.  Most  agencies  do  not  use  SGML  because  it  is  harder  to  author  in. 
Wallace's  agency  uses  it  because  it  is  much  more  robust,  and  it  is  easy  to  change  a  document  format 
to  match  customer  needs  (e.g.,  tax  law  information  for  consumer  and  for  lawyers). 

MALITZ: 

NCES  uses  PDF,  then  HTML  (optional).  They  rarely  put  entire  publication  in  HTML. 


6.   Are  there  limitations  or  specific  designations  of  software  tools  that  may  be  used  to  develop  and 

implement  web  pages  or  sites?  What  standards  are  applied  to  configuration  control  and  arrangement 
of  web-based  applications?  Do  you  have  a  direct  role  in  determining  these  standards,  or  are  they 
developed  at  an  agency  or  departmental  level?  


WALLACE: 

Their  focus  is  knowledge-based,  not  web  application.  Her  department  sets  the  standards.  They  use 
C++  and  Perl. 


H-9 


MALITZ: 


AH  publications  are  in  PDF;  all  else  in  HTML.  They  use  SQL  databases  to  support  the  web.  Malitz 
has  a  direct  role  in  determining  standards. 


7.    Are  there  general  security  standards  applied  to  the  availability  or  distribution  of  web-based 

information?  What  is  your  role  in  determining  or  implementing  these  standards?  Which  software 
products  do  you  use  to  implement  these  standards? 


WALLACE: 

The  IRS  uses  an  automated  redacting  scheme;  with  one  keystroke,  they  can  create  a  public  and 
specialized  version  of  the  same  document.  They  apply  all  security  standards  from  the  Government, 
including  SSA,  Treasury,  etc.  They  cannot  reveal  security  software. 

MALITZ: 

Only  a  few  people  who  use  developmental  server  have  access;  all  must  be  registered  users.  NCES 
uses  PPTP  encryption  for  the  developmental  server. 


8.   Has  your  agency  discussed  the  concept  of  permanent  public  accessibility  to  electronic  Government 
information  products  intended  for  public  dissemination?  How  is  your  agency  addressing  the  concerns 
of  librarians,  GPO,  and  others  for  ensuring  permanent  public  accessibility  for  electronic  Government 
information  products? 


WALLACE: 

All  tax  forms,  instructions,  publications  etc.,  are  available  for  5-6  years  online.  The  core  knowledge 
repository  maintains  material  for  14  years,  but  they  do  not  keep  every  application  back  that  far. 
IRS  can  fill  e-mail  requests  for  information  or  forms  from  earlier  years.  In  addition,  they  provide 
GOLD  CARD  SERVICES  for  librarians.  Librarians  have  their  own  page,  track  orders,  and  talk 
"live"  with  one  another.  IRS  gives  their  orders  priority. 

MALITZ: 

The  issue  of  permanent  public  accessibility  is  currently  under  discussion. 


9.    What  consideration  are  you  giving  to  creating  a  metadata  record  for  your  information  resources  or 
services  on  the  web?  (e.g.,  GILS,  MARC  or  specific  agency  locators)? 


WALLACE: 

GILS  records  are  a  subset  of  the  86  variables  that  go  into  the  core  knowledge  repository. 
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MALITZ: 


The  Dept.  of  Education  has  an  agency  locator  with  total  search  capability.  They  also  participate  in 
FedStats,  White  House  Briefing  Room,  etc. 


10.  Linda,  we  know  you  have  collected  data  on  the  comparative  costs  of  delivering  services  to  customers 
via  different  delivery  mechanisms  such  as  mail,  e-mail,  Fax  on  Demand,  kiosks,  Internet,  telephone, 
walk-in,  CD-ROM,  etc.  What  have  you  learned  about  the  costs  of  delivering  services  to  customers 
using  these  different  systems? 


WALLACE: 

Breakdown  of  comparative  costs  follows: 

a.  It  costs  IRS  $3  per  call  for  the  public  to  call  into  their  toll-free  number  and  for  IRS  to  fill  the 
request.  The  cost  to  IRS  for  the  public  to  use  the  Internet  to  access  and  use  forms  is  1  cent,  a 
difference  of  300  to  1. 

b.  The  costs  to  create  forms  on  Internet  have  gone  down,  but  the  cost  to  fill  phone  requests 
remains  the  same. 

c.  It  costs  IRS  $2.50  to  make  and  distribute  to  public  libraries  each  CD-ROM  containing  5  years 
of  tax  forms,  instructions,  and  publications.  This  is  less  than  it  takes  for  IRS  to  respond  to  one 
telephone  call.  The  D*S  also  sends  tax  CD-ROMs  to  the  depository  libraries.  They  can  mount 
them  on  their  PCs  or  allow  customers  to  check  them  out. 

d.  They  found  that  kiosks  are  very  expensive;  ATMs  are  cost-effective 
Final  Comments  from  Wallace 

The  answer  to  public  accessibility  is  not  the  Internet;  it  is  multimedia.  Delivery  mechanisms  must 
meet  the  individual  needs  of  the  customers;  no  one  size  fits  all. 
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Interview  Questions  for  Preservation  Specialists: 
Evelyn  Frangakis  and  Abby  Smith 
(November  10,  1998) 


1.  How  long  have  you  been  in  your  current  positions?  Please  describe  your  current  job  responsibilities 
and  duties.  What  portion  of  the  following  skills,  experience,  and  knowledge  do  you  use  to  perform 
your  job:  technical,  administrative,  analytical,  and  other? 

\ 

2.  What  are  the  key  problems  associated  with  digital  preservation? 

3.  What  key  policy,  organizational,  economic  and  other  non-technical  issues  need  to  be  addressed  or 
solved  to  facilitate  digital  preservation? 

4.  What  technological  strategies  or  models  have  various  organizations  such  as  the  Association  of 
Research  Libraries,  the  Digital  Library  Federation,  National  Archives,  etc.,  identified  to  address  these 
problems?  Evelyn,  one  of  the  NCLIS  staff  mentioned  that  NAL  has  established  a  structure  or 
framework  that  addresses  this  problem.  Could  you  please  talk  more  about  that?  If  you  have  any 
handouts  you  can  fax  to  us,  that  would  also  be  helpful.  Abby,  can  you  describe  some  of  CILR's 
recent  efforts  to  address  the  issue  of  digital  preservation,  including  the  survey  by  Jeff  Rothenberg  of 
the  RAND  Corporation? 

5.  What  do  we  know  about  specific  file  formats  or  mediums  that  might  facilitate  digital  preservation 
such  as  SGML,  CD-ROM,  etc.? 

6.  Are  there  any  important  preservation  issues  that  we  have  not  addressed  in  the  above-listed  questions? 
If  so,  please  discuss  them. 

7.  Could  you  please  refer  us  to  any  important  articles  on  this  topic  that  have  been  published  in  the  last 
year? 


H-12 


Summary  of  Notes  from  Conference  Call  with  Two  Preservation  Specialists: 

Evelyn  Frangakis  (NAL)  and 
Abby  Smith  (Council  on  Library  and  Information  Resources) 


1 .  How  long  have  you  been  in  your  current  position?  Please  describe  your  current  job  responsibilities 
and  duties.  What  portion  of  the  following  skills,  experience,  and  knowledge  do  you  use  to  perform 
your  job:  technical,  administrative,  analytical,  and  other? 


ABBY  SMITH  (CLIR) 

•  Been  with  CLIR  since  Sept.  1997  as  director  of  programs. 

•  Provide  program  coordination  among  the  four  areas:  economics  of  information,  leadership  in 
libraries  and  archives,  digital  libraries,  preservation  and  access. 

•  Her  primary  program  responsibility  is  in  preservation  and  access  in  libraries,  traditional  and 
digital. 

•  10  staff  members;  6  professionals,  4  admin,  support  staff. 

•  Spends  75  percent  of  time  on  policy-related  issues  and  the  remaining  25  percent  of  time  spent 
on  administrative  functions  ( i.e.,  coordinating  publications  program) 

EVELYN  FRANGAKIS  (NAL) 

•  Been  in  current  position  since  January  1997.  She  is  NAL's  first  preservation  officer. 

•  Duties:  plan,  direct,  and  implement  agency-wide  programs  for  ensuring  permanent  and  future 
accessibility  of  the  foremost  national  collection  of  materials  in  agriculture. 

•  Coordinates  activities  with  other  national  efforts,  such  as  the  U.S.  Agricultural  Information 
Network  (USAIN).  Established  in  1988,  USAIN  provides  a  forum  for  discussion  of  agricultural 
issues,  takes  a  leadership  role  in  the  formation  of  a  national  information  policy  as  related  to 
agriculture,  makes  recommendations  to  the  National  Agricultural  Library  on  agricultural 
information  matters,  and  promotes  cooperation  and  communication  among  its  members.  NAL 
participates  in  implementing  USAIN's  preservation  plan  for  print  materials,/!  National 
Preservation  Program  for  Agricultural  Literature.  The  USAIN  Preservation  Steering  Committee, 
on  which  Frangakis  serves,  oversees  this  national  cooperative  plan.  Under  the  auspices  of 
Cornell  University,  the  USAIN  plan  has  received  two  NEH  grants  to  microfilm  core  national 
and  state  agricultural  literature.  To  date,  15  states  are  participating  in  these  grants.  Other 
components  of  the  national  plan  and  program  include  determining  what  are  the  important 
archival  and  manuscript  collections  of  agricultural  materials  and  what  approaches  can  be  used 
for  their  preservation. 

•  NAL  efforts  include  developing  their  own  preservation  program  that  includes  a  traditional 
preservation  program  and  digital  efforts. 

•  Digital  efforts  are  two-pronged:  conversion  of  brittle  paper  materials  into  digital  products  by 
working  with  best  available  guidelines  to  implement  good  preservation  practices  (this  digital 
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material  will  be  available  on  the  web);  develop  a  program  to  preserve  USDA  digital  materials 
(i.e.,  materials  that  are  born  digitally). 

•  Helps  develop  preservation  policies  and  analyze  other  policies  that  come  to  NAL  or  USDA  that 
affect  preservation  of  the  collection. 

•  Time  spent  on  different  types  of  work  at  different  times.  Duties  fairly  split  among  policy, 

technical,  administrative,  analytical  areas. 

\ 

•  Staff  consists  of  two  assistants  at  present.  However,  NAL  leverages  its  preservation  resources 
by  establishing  cooperative  inter-institutional  agreements  and  contributing  funds  to  sister 
institutions  in  order  to  further  develop  the  USAIN  preservation  program  (e.g.,  cooperative 
agreements  with  Cornell  University  to  establish  copyright  clearance  for  core  historical 
literature,  developing  NEH  grant  proposals). 


2.   What  is  the  distinction  between  digital  preservation  and  permanent  public  accessibility  of  electronic 
records  (as  it  relates  to,  for  example,  the  Federal  Depository  Library  Program)?  How  long  is  "long- 
term"  preservation  vs.  "permanent  public  accessibility?" 


FRANGAKIS 

•  Some  background  from  the  USDA  perspective:  The  USDA  Digital  Publications  Preservation 
Steering  Committee  was  established  this  past  summer  to  oversee  the  implementation  of  the 
plan,  A  Framework  for  the  Preservation  of  and  Permanent  Public  Access  to  USDA  Digital 
Publications.   This  group  met  for  the  first  time  in  October  1998.  There  was  a  discussion  of 
definitions  in  order  to  place  into  context  the  universe  of  material  covered  by  theFramework. 
Publication  was  defined  as  "a  data  or  information  product  prepared  by  the  USDA  in  digital 
form  intended  to  be  disseminated  to  the  public."  The  Framework  defines  preservation  as  "the 
act  of  permanently  maintaining  and  making  available  data  or  information,  with  all  original 
content  intact." 

•  Other  experts,  such  as  Don  Waters  of  the  Digital  Library  Federation,  talk  about  preserving 
integrity  and  ensuring  persistence  of  digital  information.  The  Commission's  SGML  report  talks 
about  preservation  goals  such  as  enhancing  the  long-term  preservation  of  and  access  to 
information  of  enduring  value  for  as  long  into  the  future  as  possible. 

•  THE  CPA  Digital  Archiving  Task  Force  was  charged  to  investigate  the  means  of  ensuring 
"continued  access  indefinitely  into  the  future  of  records  stored  in  digital  electronic  form." 

•  Concept  of  preservation  in  traditional  preservation  world  examines  the  concept  of  permanence, 
but  in  the  print  world  the  concept  of  permanence  relates  to  chemical  inertness  and  mechanical 
durability.  These  concepts  do  not  translate  easily  into  a  digital  world.  In  the  digital  world,  we 
are  no  longer  dependent  on  original  copies  (i.e.,  original  copies  do  not  have  the  same  meaning.). 

•  Within  NAL,  they  use  digital  preservation  fairly  loosely  to  speak  about  both  digital  efforts: 
conversion  of  brittle  materials  and  USDA  digital  publication  preservation  efforts.  Not  sure 
professionals  in  the  library  and  preservation  community  have  a  common  understanding  of  what 
it  means,  even  though  it  is  important  to  come  to  a  common  understanding.  GPO  defines 
permanent  access  as  "Government  information  products  within  the  scope  of  the  Federal 
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Depository  Library  Program  that  remain  available  for  continuous  no-fee  public  access  through 
the  program."  The  1996  GPO  report  to  Congress,  Study  to  Identify  Measures  Necessary  for  a 
Successful  Transition  to  a  More  Electronic  Federal  Depository  Library  Program,  states  that 
"'preservation'  means  that  official  records  of  the  Federal  Government,  including  Government 
information  products  made  available  through  the  FDLP,  which  have  been  determined  to  have 
sufficient  historical  or  other  value  to  warrant  being  held  and  maintained  in  trust  for  future 
generations  of  Americans,  are  retained  by  the  National  Archives  and  Records  Administration 
(NARA)." 

•  At  NAL,  the  mission  of  its  preservation  program  is  "to  preserve  and  ensure  access  to  the 
intellectual  content  and  physical  composition  of  agricultural  works  of  national  and 
international  importance  indefinitely  into  the  future."  No  timeframe  is  mentioned  because  no 
one  at  this  time  can  say  how  long  into  the  future  information  will  be  needed. 

SMITH 

•  There  is  no  standard  accepted  method  of  ensuring  long-term  access  to  digital  information.  She 
described  preservation  goals  as  permanent  or  persistent  or  perhaps  more  accurate  to  say  that 
one  of  the  primary  goals  of  preservation  is  to  set  up  systems  that  "sustain  predictable  levels  of 
loss." 

•  Difference  between  preservation  in  a  digital  world  and  in  an  analog  world  is  that  in  a  digital 
world,  information  is  completely  independent  from  the  medium  on  which  it  is  carried.  In  an 
analog  world,  people  try  to  preserve  the  media  in  which  information  is  recorded.  No  analogy  in 
the  digital  world.  No  concept  of  preserving  the  artifact  as  an  artifact  that  has  its  own  level  of 
information. 

•  Problem  in  digital  preservation  is  that  there  is  no  way  to  ensure  that  digitally  stored 
information  can  move  from  one  software-hardware  configuration  onto  another  through 
generations.  Two  problems: 

(1)  Problem  of  instability  of  media  in  which  information  is  stored  (don't  know  how  long  CDs  or 
other  media  will  last) 

(2)  More  serious  issue  from  CLIR  point  of  view  is  that  software/hardware  configurations  on 
which  information  is  stored  become  obsolete  so  quickly  that  even  when  you  migrate 
information  from  one  system  to  another,  much  of  the  information  is  lost  (data  and 
functionality). 

•  CLIR  tries  not  to  talk  about  "digital  preservation"  but  they  cannot  avoid  it. 

•  Other  countries  may  view  preservation  differently.  England  interested  in  American  concept  of 
digital  archiving -preserving  the  integrity  of  data;  that  is,  information  is  original  and  authentic 
and  it  can  be  proved  that  data  have  not  been  changed.  According  to  Smith,  scientists  say  that 
we  will  solve  the  problem  of  authentication,  but  hasn't  been  solved  thus  far. 
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Permanent  Public  Accessibility  Issues  (FRANGAKIS) 


•  Federal  agencies  relying  on  FDLP  to  serve  an  "archiving"  function  for  retrospective  materials. 

•  Question  of  how  to  preserve  digital  information  indefinitely  into  the  future  has  not  been 
answered.  CLIR  and  NAL  are  discussing  strategies  by  opening  up  dialogue  and  promoting 
research  in  this  area. 

•  Digital  Archiving  Task  Force  Report  discussed  ensuring  the  integrity  and  long-term  availability 
of  digital  information  through  migration.  Information  on  CD-ROM  and  other  media  is  in  a 
format  that  may  or  may  not  be  readable  into  the  future  due  to  hardware/software  obsolescence. 
Even  if  media  could  be  preserved,  no  guarantee  that  it  would  be  accessible  and  functional 
indefinitely  into  the  future.  No  answers  anytime  soon. 


3.   What  are  the  key  problems  associated  with  digital  preservation? 


SMITH 

•  Additional  problems:  fragility  of  media  and  platform  dependence  issue 

(1)  Two  additional  issues:  difficulty  of  understanding  what  we  can  and  cannot  do  under 
current  copyright  law.  Latest  iteration  of  copyright  law  clarifies  copyrighting  for 
preservation  purposes,  but  still  unclear  for  access  purposes.  Library  of  Congress  is 
currently  studying  this.  Copyright  law  may  not  have  any  implications  for  Government 
information  but  many  vendors  create  derivatives  of  Government  information  and  copyright 
it.  Government  should  never  be  in  the  position  of  depending  upon  the  private  sector  to 
preserve  some  of  this  information. 

(2)  Any  transmission  link  is  as  strong  as  the  weakest  link.  The  weak  link  in  the  transmission  of 
electronic  information  is  not  technology;  it's  human  beings.  Human  infrastructure  is  not  in 
place  yet  that  would  ensure  permanent  access. 

FRANGAKIS 

•  Agrees  that  human  error  is  far  more  prevalent  than  technology  error.  Key  problems  in  digital 
preservation  are  infrastructure,  technology,  and  media.  Humans  need  to  learn  to  live,  exist, 
and  operate  in  a  digital  world.  It's  still  very  new  to  us  as  compared  to  the  print  world. 


4.    What  key  policy,  organizational,  economic,  and  other  non-technical  issues  need  to  be  addressed  or 
solved  to  facilitate  digital  preservation? 


SMITH 

•    Digital  Archiving  Task  Force  Report  met  with  consensus  in  the  community.  CLIR  has  been 
lobbying  people  to  pay  attention  to  these  issues.  No  single  community  has  stepped  forward  and 
said  this  is  our  problem  and  we  are  going  to  work  on  solving  the  problem.  Therefore,  one  of 
difficulties  is  that  organizations  that  collect,  preserve,  and  disseminate  information,  as  opposed 
to  create  information,  find  themselves  in  this  digital  world  in  which  the  preservation  of  that 


H-16 


information  must  be  thought  about  at  the  creation  stage,  not  after  the  fact.  Need  to  forge 
partnerships  with  the  computer  science  industry,  publications  industry,  scholarly  and  scientific 
publishing  communities  to  address  some  of  these  issues. 

•  One  of  perhaps  intractable  core  infrastructure  problems  is  the  issue  of  creating  a  failsafe 
archives  mechanism  for  materials  that  disappear  from  the  web.  What  happens  when 
information  is  created  and  the  people  who  created  it  do  not  have  responsibility  for  preserving 
it?  Who  is  going  to  authorize  a  failsafe  archive  that  is  going  to  take  and  preserve  that 
information  for  the  public  good?  This  would  be  the  equivalent  of  libraries,  but  so  far,  it  doesn't 
exist  and  no  one  has  expressed  interest  in  creating  it  at  the  Governmental  level. 

•  CLIR's  role  in  above:  Not  in  a  position  to  do  much  more  than  alert  people  about  the  problems. 
NAL  and  literature  that  agriculture  creates  are  one  of  few  examples  where  this  failsafe  archive 
might  work  because  NAL  is  a  national  library  dedicated  to  one  type  of  literature.  Not  the  case 
with  other  literatures  except  for  medicine  (NLM).  CLBR  is  looking  for  partners  like  ARL  to 
address  this  issue,  but  has  not  made  much  progress.  CLIR  has  been  fairly  effective  in  talking  to 
National  Science  Foundation  (NSF)  in  getting  their  second  round  of  digital  library  initiatives 
grants  to  address  the  issue  of  preservation  as  a  distinct  issue.  No  luck  with  archiving  part. 

FRANGAKIS 

Refer  to  the  report  Framework  for  the  Preservation  of  and  Permanent  Public  Access  to  USDA 
Digital  Publications  by  Paul  Uhlir,  November  1997  (listed  in  bibliography). 

•  Three  areas  of  issues: 

(1)  management  structure  and  organizational  relationships  within  and  outside  USDA, 

(2)  funding  of  program  on  a  permanent  basis  (keeping  in  mind  the  need  to  minimize  costs  of 
access  and  retrieval  to  information  users),  and 

(3)  identification  of  legislative  or  administrative  actions  or  policies  required  to  implement  a 
digital  publications  preservation  program. 

Sub-Issues:  Needs  and  Considerations  for  USDA 

•  Inventory  and  life  cycle  management:  a  comprehensive  inventory  of  all  departmental  digital 
information  products  and  how  they  are  being  managed  needs  to  be  conducted.  A  system  for 
tracking  the  creation  of  each  new  USDA  digital  information  product  that  is  intended  for  public 
distribution  needs  to  be  recommended. 

•  Technical  requirements:  identification  of  acceptable  document  formats  and  media,  and  related 
standards,  for  long-term  retention;  development  of  processes  for  transferring  all  digital 
publications  from  old  storage  media  to  new  media;  establishment  of  one  or  more  separate  back- 
up facilities  for  all  digital  publications;  review  and  establishment  of  system  security  protocols; 
review  and  establishment  of  system  interoperability  requirements;  and  identification  and 
review  of  other  permanent  digital  preservation  and  access  initiatives. 

•  User  access  and  retrieval:  provide  equitable  access  and  retrieval  services  to  all  potential  users; 
minimize  technical,  regulatory,  and  cost  barriers  to  access  and  retrieval;  assure  the  integrity  of 
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the  information  that  is  made  publicly  available;  make  the  information  as  easy  to  find  and  use  as 
possible,  with  directories  and  documentation  (metadata),  consistent  with  the  Government 
Information  Locator  System,  while  protecting  confidential  or  proprietary  information;  and 
establish  a  means  for  users  to  provide  feedback  and  a  mechanism  for  responding  to  user 
feedback. 

•    Status:  Moving  ahead  with  implementation.  USDA  CIO  accepted  the  report,  and  under  her 
guidance  NAL  established  a  national  steering  committee  made  up  of  representatives  from 
USDA  and  from  agribusiness,  research  library  community,  USAEN,  Federal  partners,  etc. 

•  Group  will  meet  on  a  quarterly  basis  for  first  2  years. 

•  Will  establish  test  groups  to  explore  issues  such  as  inventory  and  life  cycle  management, 
technical  requirements,  and  user  access  and  retrieval,  as  well  as  funding  issues. 

•  Hoping  to  get  funding  for  a  pilot  project  and  then  take  entire  framework  and  test  it  on 
an  agency  within  USDA  to  see  how  manageable  Framework  will  be  for  full-scale 
implementation. 


5.   What  technological  strategies  or  models  have  various  organizations  such  as  the  Association  of 

Research  Libraries,  the  Digital  Library  Federation,  National  Archives,  etc.,  identified  to  address  these 
problems?  Evelyn,  one  of  the  NCLIS  staff  mentioned  that  NAL  has  established  a  structure  or 
framework  that  addresses  this  problem.  Could  you  please  talk  more  about  that?  If  you  have  any 
handouts  you  can  fax  to  us,  that  would  also  be  helpful.   Abby,  can  you  describe  some  of  CILR's 
recent  efforts  to  address  the  issue  of  digital  preservation,  including  the  survey  by  Jeff  Rothenberg  of 
the  RAND  Corporation? 


SMITH 

Three  CLIR  initiatives: 

•  Commissioned  report  by  Jeff  Rothenberg  from  RAND  Corporation  on  emulation.  [Emulation  is 
the  process  of  imitating  one  system  with  another  so  both  accept  the  same  data,  execute  the  same 
programs,  and  achieve  the  same  results.]  Report  complete  and  may  be  published  by  January 
1999.  Since  report  is  highly  controversial,  CILR  will  partner  with  National  Research  Council 
to  convene  a  group  of  computer  scientists  to  engage  Rothenberg  on  issues  of  emulation  to 
stimulate  research.  Report  describes  the  weaknesses  of  migration  and  the  strengths  of 
emulation  and  sets  up  a  research  agenda  to  develop  emulation. 

•  Commissioned  an  analysis  of  migrating  file  formats  to  do  a  risk  assessment  associated  with 
those  file  formats  during  migration.  Study  commissioned  from  Cornell  using  data  from  the 
Mann  Library  (agricultural  library)  and  will  use  numeric  file  formats  and  databases  and  text 
formats.  Report  will  be  finished  by  September  1999  and  will  include  analysis  and  a  template 
that  others  can  use  for  doing  a  risk  assessment  of  migration  of  those  file  formats.  Purpose:  to 
stimulate  further  research. 
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•  Identified  a  computer  scientist  at  Carnegie-Mellon  University  (CMU),  John  Ockerbloom,  who 
has  developed  a  system  of  file  conversion;  type  of  migration  that  converts  web-based  materials 
to  different  file  formats,  called  TOM  (Typed  Object  Model). 

(www.cs.cmu.edu/afs/cs.cmu.edu/user/spok/www/defense/index.html).  He  developed  this  as  part  of 
his  thesis.  Working  with  CMU  to  see  if  they  can  bring  his  concepts  into  fuller  application  to  do 
an  assessment  about  its  scalability. 

•  Log  on  to  publications  on  CLIR  site,  which  summarize  Rothenberg  report.  Water's  report 
addresses  definition  of  digital  preservation. 


6.    What  do  we  know  about  specific  file  formats  or  mediums  that  might  facilitate  digital  preservation 
such  as  SGML,  CD-ROM,  etc.? 


SMITH 

Nothing  to  say  about  this 
FRANGAKIS 

•    For  conversion  efforts  from  paper  to  digital  images,  SGML  serves  as  an  important  descriptive 
markup  tool.  Thinks  it  will  be  valuable  to  them.  CD-ROM  serves  specific  functions  in  NAL's 
Preservation  Program  but  right  now  has  a  limited  life  expectancy.  NAL  is  looking  for  things 
that  are  non-proprietary,  platform  independent,  things  that  will  allow  user  full  access  to  the 
content  of  digital  products.  They  know  that  media  will  continue  to  change. 


7.   Are  there  any  important  preservation  issues  that  we  have  not  addressed  in  the  above-listed  questions? 
If  so,  please  discuss  them. 


No. 


8.    Could  you  please  refer  us  to  any  important  articles  on  this  topic  that  have  been  published  in  the  last 
year? 


1.  Margaret  Hedstrom  at  the  University  of  Michigan  School  of  Information  believes  that  there  is  a 
reliable  way  of  preserving,  with  predictable  levels  of  loss,  migration  of  digital  information, 
through  ASCII.  It  happens  now.  She  is  a  leading  authority  in  the  field. 

2.  Check  ARL's  website. 

3.  Coalition  of  Networked  Information  -  Dedicated  to  computer  use  in  education.www.cni.org 

4.  White  paper  on  access  authorization.  Developing  infrastructure  for  digital  libraries. 

5.  www.RLG.org/preserv  -  (includes  information  onHedstrom's  research). 

6.  Reference  Model  for  Open  Archival  Information  Systems: 

http://ssdoo.gsfc.nasa.gov/nost/isoas/ref_model.html 

White  Book,  issue  #4,  Sept.  1998-  preservation  of  digital  information;  technical 
recommendation  for  use  in  developing  consensus  on  what's  required  of  any  archive  to  provide 
permanent  preservation.  Hoping  to  turn  this  into  an  ISO  standard,  but  now  just  in  draft  form. 
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Interview  Questions  for  Information  Resources  Specialist: 

John  Bertot 
(November  18, 1998) 


1 .  What  are  the  primary  obstacles  to  successful  information  resources  management  (IRM)  practices  in 
the  Federal  Government,  in  priority  order?  What  changes  should  occur  to  eliminate  or  alleviate  the 
barriers? 


2.  In  your  article  on  the  impact  of  Federal  IRM  on  agency  missions,  you  mention  the  reinvention  of 
IRM  to  be  the  key  link  between  agency  information  and  agency  performance.  Can  you  describe  a 
small  and  a  large  Federal  agency  that  currently  meet  this  goal  in  spite  of  the  lack  of  a  concentrated, 
coordinated  Federal  IRM  policy?  Why  are  they  more  effective  than  other  agencies? 

3.  We  have  found  in  our  interviews  with  Federal  agency  personnel  that  many  agencies  have  not  come  to 
terms  with  two  important  issues:  information  life-cycle  management,  and  the  concept  of  permanent 
public  availability  of  electronic  Government  information.  What  are  some  of  the  larger  policy  issues 
that  have  prevented  agencies  from  addressing  these  important  issues? 

4.  In  our  site  visits  to  Federal  depository  libraries  in  the  D.C.  metropolitan  area,  we  are  keenly  aware 
that  the  problems  and  issues  faced  by  FDLs  here  are  different  than  they  might  be  for  FDLs  located  in 
more  isolated,  rural  areas.  Based  on  your  survey  of  public  libraries  connected  to  the  Internet,  what 
are  some  of  the  key  concerns  or  problems  faced  by  users  who  want  to  access  electronic  Government 
information  who  live  in  small,  isolated  communities  with  limited  resources? 

5.  Based  on  your  experience  in  working  with  Federal  Government  agencies  that  are  analyzing  their  web 
usage,  what  are  the  key  questions  they  want  to  answer  and  how  are  they  using  the  data?  Are  they 
analyzing  the  websites  for  technical  or  content-related  purposes?  What  techniques,  other  than  log 
file  analysis,  are  being  used?  What  agency  units  or  departments  (e.g.,  IT,  program  areas,  CIO)  are 
involved  in  the  process? 
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Interview  Questions  for  Information  Resources  Specialist: 
Charles  McClure 
(November  24, 1998) 

1.  What  is  the  current  status  of  IRM  policy  since  your  1994  article,  "Federal  Information  Resources 
Management:  New  Challenges  for  the  Nineties?"  Specifically,  has  OMB  or  another  appropriate 
agency  begun  addressing  the  issue  of  developing  a  broad  vision  that  reflects  the  evolving  role  of  IRM 
within  the  Government  with  general  guidelines  and  standards  for  all  Federal  agencies  to  better 
manage  the  life  cycles  of  information?  If  so,  how? 

2.  We  have  found  in  our  interviews  with  Federal  agency  personnel  that  many  agencies  have  not  dealt 
with  two  important  issues:  information  life-cycle  management,  and  permanent  public  accessibility  of 
electronic  Government  information.  What  are  some  of  the  larger  planning,  policy,  and  organizational 
issues  that  are  preventing  agencies  from  addressing  these  important  problems? 

3.  Could  you  talk  a  little  more  about  the  design-based  assessment  for  evaluating  Government  websites 
that  you  described  in  the  1997  Proceedings  of  the  60th  ASIS  annual  meeting?  What  were  the 
technical  and  policy  problems  in  the  design-based  assessment?  What  specific  policy  issues  did  you 
assess? 


4.  What  is  the  status  of  electronic  record  management  (ERM)  guidance  for  Federal  agencies  since  the 
1998  conference?  Is  there  another  conference  planned  next  year?  What  specific  guidance  are  the 
NARA  Working  Group  and  other  agencies  planning? 

5.  What  would  you  say  are  the  top  three  Federal  IRM  challenges  in  the  next  decade? 
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Summary  of  Notes 
Interview  with 
John  Bertot,  Associate  Professor,  SUNY/Albany 


1.    What  are  the  primary  obstacles  to  successful  IRM  practices  in  the  Federal  Government,  in  priority 
order?  What  changes  should  occur  to  eliminate  or  alleviate  the  barriers? 

V 

•  IRM  is  not  on  the  radar  for  top-level  agency  managers  and  it  will  never  be  raised  up  to  the 
point  of  where  it  matters. 

•  IRM  has  been  lost  in  the  transition  to  the  CIO.  CIO  is  the  next  iteration  of  IRM.  There  has 
been  20  years  worth  of  talking  and  it  has  never  seemed  to  make  it  out  of  the  administrative 
trenches  of  the  agencies.  Typically,  IRM  has  been  a  low-level  position  located  within  the 
printing,  reprographic,  or  records  management  units  of  agencies.  IRM  is  not  viewed  as 
strategic  or  long-range  function. 


2.  In  your  article  on  the  impact  of  Federal  IRM  on  agency  missions,  you  mention  the  reinvention  of 
IRM  to  be  the  key  link  between  agency  information  and  agency  performance.  Can  you  describe  a 
small  and  a  large  Federal  agency  that  currently  meets  this  goal  in  spite  of  the  lack  of  a  concentrated, 
coordinated  Federal  IRM  policy?  Why  are  they  more  effective  than  other  agencies? 


The  article  is  based  on  Bertot's  dissertation. 

The  purpose  of  the  survey  was  to  get  an  internal  assessment  on  what  IRM  is  trying  to  do,  and  to  get 
an  external  assessment  of  what  IRM  is  doing.  Bertot  tried  to  compare  the  two  in  relation  to 
strategic  planning. 

Generally,  those  agencies  that  understood  IRM  tended  to  be  the  smaller  agencies.  There  is  a  scale 
factor,  and  much  more  attention  was  given  to  IRM  in  the  smaller  to  mid-sized  agencies.  FDIC  and 
the  Peace  Corps  were  doing  some  interesting  things. 

Other  factors  relating  to  size: 

Small  to  medium-sized  agencies  have  fewer  programs  and  staff;  with  fewer  administrative 
layers,  there  are  fewer  communication  and  organizational  barriers. 

o       One  can  more  easily  work  collaboratively  in  a  small  organization. 

•  Top  administrators  are  not  as  removed  from  day-to-day  operations  and  can  ideally 
participate  more  in  implementing  new  initiatives  because  they  have  a  vested  interest  in  the 
projects'  working. 

•  There  is  less  oversight  from  OMB  and  Congress  for  smaller  agencies.  The  smaller  agencies 
tend  to  have  less  mandated  legislation  that  can  interrupt  work,  so  they  tend  to  have  higher 
motivation  to  finish  projects. 
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•  Larger  agencies  tend  to  have  the  greater  expertise.  Smaller  agencies  may  have  better  levels 
of  management,  but  they  do  not  always  have  the  experts. 

Models 

•  As  far  as  larger  agencies  were  concerned,  Treasury  was  moving  along.  However,  many  have 
a  central  agency  component  that  is  not  very  powerful,  althoughsubagencies  might  be  very 
powerful.  For  example,  Treasury  has  IRS  and  the  FBI,  and  they  are  pretty  powerful  players. 

•  Another  agency  that  has  done  a  great  deal  of  work  with  IRM  is  EPA.  EPA,  along  with  AID, 
are  strange  models.  They  have  large  data  shops,  but  they  are  all  contractors.  The  model  that 
is  adopted  for  information  and  information  technology  management  makes  a  difference. 
Whether  IRM  is  in-house  or  out-sou  reed  has  a  real  impact  on  how  it  is  implemented  inside 
the  agency,  and  the  choice  of  contractor  really  matters.  Another  aspect  that  makes  EPA 
unique  is  that  a  large  portion  of  their  system  management  function  occurs  in  North  Carolina. 


3.  We  have  found  in  our  interviews  with  Federal  agency  personnel  that  many  agencies  have  not  come  to 
terms  with  two  important  issues:  information  life  cycle  management,  and  the  concept  of  permanent 
public  availability  of  electronic  Government  information.  What  are  some  of  the  larger  policy  issues 
that  have  prevented  agencies  from  addressing  these  important  issues? 


•  The  biggest  barrier  to  successful  implementation  of  IRM  is  that  agencies  do  not  view 
information  as  a  resource.  There  is  little  or  no  understanding  of  the  concept  of  information  as 
a  life  cycle;  it's  not  linear. 

•  IRM  policy  initiatives  and  legislation  do  not  fully  address  the  life  cycle  of  information.  It  is 
mentioned  in  some  of  the  policies  developed  within  the  last  20  years,  but  not  adequately 
addressed.  Most  policy  initiatives  focus  on  the  technology  side  of  the  issue,  probably  because 
it  is  tangible. 

•  The  web  has  created  problems  that  have  not  been  handled.  Many  agencies  believe  that  if  it  is 
up  on  the  web,  it  has  been  published.  Along  with  the  pressure  over  Title  44  Reform,  there  is 
no  discussion  of  preservation  and  public  accessibility.  Should  we  move  to  an  electronic  FDL 
program?  What  does  that  mean  and  how  will  that  work? 

•  GPO  is  under  attack  for  being  deficient  in  distributing  Government  information  to  the 
public.  One  reason  is  because  GPO  (centralized  print  environment)  is  so  slow,  and  the 
technology  allows  distribution  to  be  handled  more  efficiently  (decentralized,  electronic 
environment).  Agencies  are  under  the  gun  to  cut  costs,  so  by  putting  information  on  their 
websites  and  contracting  printing  jobs  with  outside  sources,  they  don't  have  to  go  through 
GPO. 

•  Going  electronic  does  offer  potential.  The  back  end  means,  however,  that  anyone  with  access 
is  a  vehicle  for  getting  Government  information.  Putting  information  up  on  the  web  does  cut 
costs,  but  we  have  not  figured  out  a  systematic  way  of  distributing  Government  information 
to  the  public,  making  sure  it  is  preserved  for  posterity  and  provided  to  the  public  on  a  long- 
term  basis. 
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4.  In  our  site  visits  to  Federal  Depository  Libraries  in  the  D.C.  metropolitan  area,  we  are  keenly  aware 
that  the  problems  and  issues  faced  by  FDLs  here  are  different  than  they  might  be  for  FDLs  located  in 
more  isolated,  rural  areas.  Based  on  your  survey  of  public  libraries  connected  to  the  Internet,  what 
are  some  of  the  key  concerns  or  problems  faced  by  users  who  want  to  access  electronic  Government 
information  who  live  in  small,  isolated  communities  with  limited  resources? 


Recently  Bertot  conducted  some  research  in  rural  Pennsylvania  to  study  public  libraries.  Public 
libraries  in  rural  communicates  face  large  problems  with  access  and  technology. 

•  These  areas  are  composed  of  populations  that  tend  not  to  have  computers  in  the  home,  so 
they  rely  totally  on  the  library  for  Internet  access. 

•  The  public  libraries  tend  to  only  have  one  station  in  these  rural  areas. 

•  Computers  are  slow;  libraries  have  56  K  modems,  but  they  do  not  necessarily  have  access  to  a 
56  K  Internet  provider. 

•  Patrons  can  only  reserve  computers  in  half-hour  time  slots. 

•  Libraries  may  or  may  not  have  access  to  print  equipment. 

•  Staff  training  is  minimal  due  to  cost  and  little  access  to  computers.  (Models  like  GPO  Access 
have  been  useful,  but  patrons  and  librarians  still  need  one  site  for  access  to  all  Government 
agencies  rather  than  many  individual  websites.  Most  agencies  have  more  than  one  site.) 

•  Staff  are  competing  with  patrons  for  access  because  there  are  only  one  or  two  computers. 

•  Libraries  cannot  always  pay  for  printing  services.  Some  are  passing  the  cost  to  the  user,  but 
they  are  trying  to  avoid  that  approach  (e.g.,  first  5  pages  are  free  and  then  it  is  10  cents  per 
page). 


5.  Based  on  your  experience  in  working  with  Federal  Government  agencies  that  are  analyzing  their  web 
usage,  what  are  the  key  questions  they  want  to  answer  and  how  are  they  using  the  data?  Are  they 
analyzing  the  websites  for  technical  or  content-related  purposes?  What  techniques,  other  than  log  fde 
analysis,  are  being  used?  What  agency  units  or  departments  (e.g.,  IT,  program  areas,  CIO)  are 
involved  in  the  process? 


Bertot  has  not  seen  agencies  doing  very  much  with  their  web  statistics.  Part  of  the  reason  is 
that  they  do  not  want  to  make  the  information  public.  For  example,  one  agency  had  a  request 
for  all  of  their  agency  log  file  records.  They  panicked  and  rejected  the  request  on  the 
condition  of  privacy.  If  one  can  access  the  log,  one  can  get  IP  addresses,  and  they  were  afraid 
that  someone  would  use  this  information  as  a  reverse  directory  mailer. 

A  second  reason  is  that  sometimes  it  is  difficult  to  get  the  statistics  if  a  different 
administrative  unit  within  the  agency  is  managing  the  website.  They  won't  necessarily  turn 
them  over  to  the  unit  that  needs  the  statistics  because  it  crosses  administrative  barriers. 
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A  third  reason  relates  to  records  management  of  log  files.  Should  we  "schedule"  log  files  for 
NARA?  NARA  doesn't  want  this  to  happen  either.  They  would  then  have  to  schedule  the 
information  for  retention.  However,  this  raises  the  question  of  whether  the  logs  are  public 
information. 

It  was  not  until  recently  that  there  was  a  demand  to  look  at  web  statistics  as  a  management 
and  strategic  decision-making  tool.  Managers  are  still  learning  how  to  use  them.  Right  now, 
these  statistics  are  used  primarily  by  network  and  system  administrators. 

Bertot  wonders  how  many  agencies  are  doing  web  analysis  and  evaluation  given  Circular 
130-A,  which  cautions  agencies  not  to  do  so  if  it  creates  a  paperwork  burden  for  them. 
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Summary  of  Notes 
Interview  with  Charles  McClure, 
Distinguished  Professor,  School  of  Information  Studies,  Syracuse  University 


1.    What  is  the  current  status  of  IRM  policy  since  your  1994  article,  "Federal  Information  Resources 
Management:  New  Challenges  for  the  Nineties"?  Specifically,  has  OMB  or  another  appropriate 
agency  begun  addressing  the  issue  of  developing  a  broad  vision  that  reflects  the  evolving  role  of  IRM 
within  the  Government  with  general  guidelines  and  standards  for  all  Federal  agencies  to  better 
manage  the  life  cycles  of  information?  If  so,  how? 


IRM  policy  came  and  went  and  no  one  noticed: 

GSA  is  mounting  its  CIO  university  effort  to  provide  education  and  training  to  CIOs. 

•  IRM  in  Government  policy  is  now  whole  world  to  CIO. 

•  Many  agencies  do  not  now  know  what  to  do  with  IRM. 

•  ITMRA  (Information  Technology  Management  Reform  Act  of  1996) — McClure  thought  this 
policy  would  strengthen  ERM,  but  in  reality,  it  took  responsibility  away  from  existing  IRM 
people  and  gave  it  to  CIO;  it  gave  more  attention  to  technology  management. 

•  A  few  agencies  don't  know  what  to  do  with  IRM  staff  since  CIO  is  on  board;  in  other 
agencies  there  is  conflict  between  IRM  and  CIO  functions. 

•  Eighty-two  percent  of  technology  efforts  in  agencies  are  currently  focused  on  Y2K  efforts. 


2.    We  have  found  in  our  interviews  with  Federal  agency  personnel  that  many  agencies  have  not  dealt 
with  two  important  issues:  information  life  cycle  management,  and  permanent  public  accessibility  of 
electronic  Government  information.  What  are  some  of  the  larger  planning,  policy,  and  organizational 
issues  that  are  preventing  agencies  from  addressing  these  important  problems? 


•  There  is  no  staff  or  time  to  devote  to  standards  and  interoperability. 

•  Even  if  agencies  had  staff  and  time,  staff  need  to  upgrade  skills  and  knowledge. 

•  Information  life  cycle  and  permanent  public  accessibility  are  not  priorities  for  agencies;  they 
don't  seem  to  understand  the  issues. 

•  For  example,  GPO  Reform  Bill  is  dead  in  the  water;  no  one  in  Congress  cared  about  it. 

•  No  one  is  concerned  about  long-term  accessibility. 
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3.    Could  you  talk  a  little  more  about  the  design-based  assessment  for  evaluating  Government  websites 
that  you  described  in  the  1997  Proceedings  of  the  60th  ASIS  annual  meeting?  What  were  the 
technical  and  policy  problems  in  the  design-based  assessment?  What  specific  policy  issues  did  you 
assess? 


They  are  now  using  more  advanced  website  methodology  (i.e.,  a  4-legged  approach): 

•  User-based:  Usability  testing;  simulates  user  searching  that  is  videotaped.  With  fairly 
sophisticated  graduate  students,  they  use  scripted  search  analysis  with  a  range  of  criteria. 
System  and  design  staff  are  showing  videotapes  to  so  they  can  see  the  problems  with 
searching  information  on  specific  sites.  Agencies  have  used  different  audiences  to  do  testing 
based  on  objectives  and  purpose  of  sites. 

•  Log  analysis:  Using  in-house  scripts  beyond  WebTrends  and  Log  Tracker  that  allow  them  to 
do  cross-file  analysis  with  access  vs.  error  and  browser  files.  Perl  scripts  allow  them  to  dump 
selected  variables  in  log  files  into  SASS  or  SPSS.  Commercial  products  do  not  do  cross-log 
analysis  well. 

•  Policy  analysis:  Internal  policies  (who's  in  charge)  and  external  policies  (e.g.,  Freedom  of 
Information  Act,  public  access,  privacy  issues,  etc.). 

•  Management  assessment:  How  is  agency  department  organized  for  web  maintenance  and 
evaluation?  Information  is  gathered  through  interviews  and  focus  groups  with  managers. 


4.    What  is  the  status  of  electronic  records  management  (ERM)  guidance  for  Federal  agencies  since  the 
1998  conference?  Is  there  another  conference  planned  next  year?  What  specific  guidance  are  the 
NARA  Working  Group  and  other  agencies  planning? 


•  Update:  McClure  and  Tim  Spreche  are  working  on  a  new  project,  PARS-  Public  Access 
Rating  System.  The  purpose  of  this  project  is  to  create  a  core  set  of  performance  measures 
and  indicators  (now  being  developed  for  four  Government  agencies)  with  public  access 
criteria  to  help  agencies  rate  their  websites.  Agencies  will  be  able  to  determine  the  degree  to 
which  the  site  is  publicly  accessible.  It's  difficult  to  sell  ERM  by  itself,  so  they  are  taking  a 
public  access  approach. 

•  National  Archives  hasn't  done  much.  Court  ruling  delayed  6-8  months. 


5.    What  would  you  say  are  the  top  three  Federal  IRM  challenges  in  the  next  decade? 


•  IT  management  policy  development  is  on  hold  due  to  Y2K.   No  one  currently  knows  how 
good  or  bad  preparation  is  for  this. 

•  How  best  to  integrate  and  coordinate  IT  and  IT  management.  Agencies  do  not  have  a  good 
handle  on  this  yet. 
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•  Issues  of  interoperability  and  standards  that  cut  across  all  agencies.  Need  to  be  able  to  access 
Government  information  horizontally  rather  than  vertically.  (For  example,  for  public  access 
sites;  GILS,  gov.doc  1,  and  one  more;  no  way  public  can  access  specific  information  from  one 
point  of  entry.  GILS  does  not  work  the  way  it  was  originally  conceived.) 

•  Lack  of  money  for  training  and  education.  ERM  graduate  students'  degrees  are  useful  for 
about  1-2  years.  After  that,  their  skills  are  50  percent  out  of  date.  Government  agencies  have 
well  meaning  people  who  don't  have  the  knowledge  and  skills  to  implement  policies.  For 
example,  agencies  say  they  don't  need  to  send  copies  of  all  products  to  GPO  because  they  are 
on  their  website.  Then  you  ask  them  will  it  be  there  in  6  months  and  they  have  given  no 
thought  to  this  issue. 
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Appendix  I 


Sample  Agency  Meeting  Agenda 
Electronic  Government  Information  Products  Assessment 
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Sample  Agency  Meeting  Agenda 
Electronic  Government  Information  Products  Assessment 


Meeting  with  Health  and  Human  Services  Agency 
Representatives  and  Product  Respondents 

Tuesday,  September  15, 1998 
9:15  a.m. 


9: 1 5        Introductions  Westat,  NCLIS,  and  HHS  Personnel 

9:25        Background  and  Purpose  of  the  Meeting  Westat  and  NCLIS 

9:30        General  Questions  and  Answers  Westat,  NCLIS,  HHS  Personnel 

•  Sections  A  and  B  on  the  Questionnaire 

10:00      Q  and  A  to  Sections  C  and  D  NCLIS,  Westat,  HHS  Personnel 

•  Review  Data  Collection  Procedures 


10:30  Break 

10:45      Agency  Discussion  Questions  Westat  and  HHS  Personnel 

(Discussion  questions  posted  on  NCLIS  web  site) 

1 1 :45  Adjournment 
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Appendix  J 
Task  16 

Assessment  of  Electronic  Government  Information  Products 

Statement  of  Work 
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Section  J:  Background  and  Objectives 

Background 

An  "Assessment  of  Electronic  Government  Information  Products"  (hereinafter  referred  to  as  the 
"Assessment")  is  a  research  study  being  conducted  through  an  interagency  agreement  between 
the  National  Center  for  Education  Statistics  (NCES)  and  the  U.  S.  National  Commission  on 
Libraries  and  Information  Science  (NCLIS)  on  behalf  of  the  Superintendent  of  Documents,  U.S. 
Government  Printing  Office  (GPO),  pursuant  to  an  Interagency  Agreement  between  NCLIS  and 
GPO,  approved  by  the  Joint  Committee  on  Printing  (JCP). 

Information  gathered  from  this  assessment  is  to  be  used  by  the  Superintendent  of  Documents  to 
facilitate  improved  public  access  to  Federal  Government  information  made  available  to  Federal 
depository  libraries  and  the  general  public  through  the  Federal  Depository  Library  Program 
(FDLP).  The  assessment  will 

(1)  identify  medium  and  format  standards  that  are  the  most  appropriate  for  permanent  public 
access; 

(2)  assess  the  cost-effectiveness  and  usefulness  of  various  alternative  medium  and  format 
standards;  and 

(3)  identify  public  and  private  medium  and  format  standards  that  are,  or  could  be  used  for 
products  throughout  their  entire  information  life  cycle,  not  just  at  the  dissemination  or 
permanent  public  access  phase. 

Objectives 

The  contractor  shall  undertake  research  and  data  collection  from  Federal  agencies  in  all  three 
Branches,  as  well  as  solicit  the  opinions  of  selected  knowledgeable  experts.  The  contractor  shall 
also  complete  an  analysis  of  the  data  and  opinions  for  the  purpose  of  interpreting  their  general 
meaning  and  significance,  including  identifying  broad  emerging  trends  and  patterns,  and 
documenting  findings,  conclusions  and  recommendations  in  a  deliverable  final  report. 

More  specifically,  for  a  cross-section  of  Government  information  products,  the  goals  are  to:  (1) 
determine  in  which  format(s)  and  medium(s)  such  products  are  now  produced,  using  which 
standard(s)  if  any  (whether  promulgated  by  official  standards-setting  bodies  such  as  ISO,  NISO, 
ANSI  or  FIPS,  or  voluntarily  adopted  through  common  agency  practice,  such  as  the  use  of  file 
formats  such  as  PDF  or  TIFF,  or  Microsoft  Word  or  dBase);  and  (2)  assess  agency  future  plans 
for  new  or  changed  products,  including  the  medium(s)  and  format(s)  in  which  they  will  be 
disseminated  for  permanent  public  access. 

The  Superintendent  of  Documents  will  use  the  results  of  this  work  effort  to  continue  to  plan  and 
implement  the  transition  to  a  more  electronic  FDLP.  The  5  major  specific  objectives  are: 
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>  First,  with  respect  to  electronic  publishing  practices  and  plans  of  Federal  agencies  (including 
ways  in  which  the  FDLP  can  best  accommodate  them),  the  objective  is  to  provide  an  analysis 
of  current  practices  as  well  as  future  plans  for  creating,  disseminating,  and  providing 
permanent  public  accessibility  to  electronic  information  products,  and  to  identify  the 
standards  for  software,  and  electronic  mediums  and  formats  that  are  or  will  be  used 
throughout  the  product's  information  life  cycle,  from  creation  to  archiving,  but  especially  by 
the  time  of  the  permanent  public  accessibility  phase; 

>  Second,  with  respect  to  cost  effectiveness  of  various  dissemination  mediums  and  formats  that 
are,  or  could  be,  utilized,  the  objective  is  to  gather  information  on  standards  (whether 
mandated  or  consensual)  that  will  assist  the  FDLP  in  making  near-term  decisions  regarding 
the  cost-effectiveness  of  alternative  mediums  and  formats  for  all  FDLP  participants.  This 
information  should  also  assist  participants  in  long-term  planning  for  permanent  public 
accessibility,  and  the  collection  and  analysis  of  overall  information  life-cycle  costs; 

>  Third,  with  respect  to  the  practical  utility  of  various  electronic  mediums  and  formats  to 
depository  libraries  and  the  public,  the  objective  is  to  identify  preferred  standards  used  in 
various  mediums  and  formats  that  depository  libraries  will  need  to  support; 

>  Fourth,  with  respect  to  utilizing  standards  employed  in  mediums  and  formats  that  can  be  used 
throughout  all  stages  of  the  information  life  cycle  (including  creation,  composition,  computer 
terminal  display,  encryption,  secure  digital  signature  with  non-repudiation  and  secure 
transmission  capabilities),  electronic  dissemination,  but  especially  permanent  public 
accessibility,  the  objective  is  to  assess  standards  for  basic  security  services  in  order  to 
provide  for  secure  and  reliable  transmission  and  document  interchange;  and 

>  Fifth,  with  respect  to  standards  that  are  being  developed  and  used  in  the  private  sector,  the 
objective  is  to  identify  existing  and  planned  standards  for  the  purpose  of  determining  what 
the  FDLP  must  do  to  accommodate  their  adoption  by  the  agency  in  terms  of 
hardware/software  requirements,  staff  and  user  education  and  training,  and  budgetary 
impacts. 

For  the  purposes  of  this  survey  an  agency  Web  site  is  considered  an  electronic  information 
service,  and  one  or  more  products  may  reside  on  the  service.  Web  sites,  per  se,  are  not 
considered  individual  Government  information  products. 
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Section  II:  Subtasks 


The  contractor  will  be  provided  background  materials  to  assist  in  conducting  the  required  work, 
including  a  suggested  data  collection  instrument,  examples  of  completed  forms,  selection  criteria 
for  finalizing  the  list  of  agencies  and  products  to  be  surveyed,  and  additional  materials  (see 
Appendix  C).  Working  with  representatives  of  NCLIS  and  GPO,  the  contractor  shall  undertake 
the  following  specific  activities  and  complete  them  by  the  scheduled  dates  shown  below.  A 
detailed  explanation  of  each  activity  appears  in  Appendix  C: 


Activity  No. 

Activity  Title 

1 

Prepare  a  Plan  of  Action 

2 

Develop  a  Data  Collection  Plan  identifying  preliminary  agency  and  product 
selections,  and  a  schedule  of  interviews,  focus  group  meetings  and  site  visits, 
using  Appendix  B  as  a  guide 

3 

Conduct  interviews,  focus  group  meetings  and  site  visits;  revise  preliminary 
agency  and  product  selections  if  necessary 

4 

Pre-test  the  Data  Collection  Instrument  (Appendix  A);  make  changes  if 
necessary 

5 

Create  a  Baseline  Inventory  Products  Data  Base  ready  to  populate  with  actual 
data 

6 

Collect  the  data  from  Federal  agencies,  experts,  and  literature  references 

7 

Prepare  tabulations  and  summaries  based  on  the  populated  data  base  and 
analysis 

8 

Recollect  and  retabulate  if/as  necessary 

9 

Prepare     a     draft     Report     documenting     findings,     conclusions  and 
recommendations,  and  provide  oral  briefings  on  request;  obtain  feedback  from 
Government 

10 

Prepare  a  final  Report  and  submit  to  the  Government 
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Section  III.  Deliverables  And  Key  Events  Schedule 


Deliverables,  in  terms  of  their  delivery  in  weeks  after  award,  are  shown  below.  For  each  of  the 
deliverables,  the  contractor  shall  provide  six  copies  to  the  COTR.  [This  is  the  original  schedule 
and  does  not  reflect  the  changes  that  were  later  negotiated  between  NCLIS  and  Westat.] 


^A/ppl<r<i  Aftpr  AwarH 

VV  LvlvO  /  \  1  1^1    /  iWulU 

Government-contractor  planning  meeting 

1 

Plan  oi  Action  approved 

2 

Approval  of  prelim,  agency/contact  list 

3 

Completion  of  interviews/meetings/site  visits 

12 

Approval  of  Data  Collection  Plan 

12 

Agency  data  collection  instruments  transmitted  to  agencies 

13 

Actual  data  collected 

16 

First  oral  briefing 

17 

Tabulations/summaries  completed 

17 

Recollection  and  retabulation  completed 

19 

Second  oral  briefing 

19 

Draft  Report  submitted 

20 

Final  oral  briefing 

22 

Final  Report  submitted 

25 
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Section  IV  -  Management  Reporting  Requirements 

The  contractor  shall  provide  the  following  reports  for  the  purpose  of  maintaining  a  detailed 
record  of  work  ordered  and  funds  spent  on  this  task  order: 

Cost  Proposal; 

Schedule  of  Deliverables 

Task  Order  "Log"  or  menu  [monthly]; 

Monthly  Cost  Report; 

Task  Invoices;  and 

Reimbursement  Report  [monthly]. 

Details  pertaining  to  these  and  ancillary  reporting  activities  follow. 

Cost  Proposals 

The  contractor  shall  provide  a  cost  proposal  to  accompany  the  contractor's  response  to  the  work 
identified.  The  cost  proposal  will  be  reviewed  and  approved  by  the  COTR  and  the  contracting 
officer,  who  will  communicate  the  approval  to  the  contractor.  [Should  there  be  a  need  to  modify 
the  cost  proposal,  the  COTR  and  the  contracting  officer  will  communicate  that  instruction  to  the 
contractor's  Project  Director,  who  shall  submit  a  revised  cost  proposal  for  approval.] 

Task  Order  "Log" 

The  contractor  shall  provide  a  monthly  report  indicating  the  tasks  ordered  by  the  COTR  and  the 
contracting  officer  to  date  and  showing  the  internal  (to  the  contractor)  tracking  number  assigned 
to  each.  The  contractor  shall  insure  that  this  report  includes  such  summary  information  as:  the 
contractor  staff  member  assigned  as  task  leader;  date(s)  associated  with  the  task,  including  1) 
date  assigned  to  the  contractor;  2)  date  scheduled  for  completion  of  the  task;  and  3)  upon 
completion,  date  of  invoice(s)  for  services  rendered  in  support  of  the  task. 

Monthly  Cost  Report 

The  contractor  shall  provide  a  monthly  report  to  the  cotr  and  the  contracting  officer  as  specified 
under  the  terms  and  conditions  of  the  contract  and  detailed  in  the  proposal.  As  part  of  the 
monthly  report,  the  contractor  shall  include  a  list  of  deliverables  and  their  anticipated  due  dates. 

Reimbursement  Report 

The  contractor  shall  provide  the  COTR  and  the  contracting  officer  with  a  status  report  on 
reimbursement  activities.  The  contractor  shall  insure  that  the  report  includes  the  names  of  the 
consultants  and  participants  and  any  others  who  are  to  be  reimbursed  for  expenses  incurred,  as 
applicable  and  approved  by  the  COTR  and  the  contracting  officer.  The  contractor  shall  include: 
1)  the  subtask  for  which  the  person  is  subject  for  reimbursement;  2)  the  amount  subject  to 
reimbursement  for  each  subtask  in  which  the  person  is  a  participant;  3)  the  date  each  participant 
submitted  the  necessary  reimbursement  form  for  each  subtask;  4)  the  date  the  participant  is 
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scheduled  to  be  issued  reimbursement  for  each  subtask;  and  5)  cumulative  reimbursement  totals 
to  date,  updated. 

Period  of  Performance 

The  period  of  performance  for  this  task  is  eight  months  from  the  date  the  award  is  signed. 
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